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Abstract 

Metrics  were  developed  to  assess  and  monitor  in  a  statistically  valid  manner  the 
development  of  good  governance  in  Afghanistan.  Measures  of  effectiveness  (MOE)  were 
developed  for  civil-military  operations  in  Afghanistan.  This  report  documents:  1) 
methods  to  develop  MOE,  which  may  be  used  to  the  develop  MOE  for  any  type  of 
operation;  2)  the  civil-military  end  state  defined  in  the  process;  3)  a  comprehensive 
system  of  MOE  for  Afghanistan;  4)  a  new  assessment  tool,  the  Assessment  of 
Governance  Quality  Indicators,  which  allows  assessment  and  monitoring  of  governance 
development;  and  5)  a  national  sampling  strategy  to  monitor  governance  development 
throughout  Afghanistan.  Methods  and  metrics  documented  here  have  applicability  for 
both  the  military  and  the  civilian  assistance  community,  as  well  as  for  operations  in  other 
countries.  This  report  represents  work  performed  primarily  by  the  staff  of  the  Coalition 
Joint  Civil-Military  Operations  Task  Eorce  (CJCMOTE)  in  support  of  Operation 
Enduring  Ereedom  in  Afghanistan  during  the  period  July  2003  through  January  2004. 

The  work  involved  close  cooperation  between  CJCMOTE,  other  elements  of  the 
Coalition  military,  civilian  US  Government  Agencies,  and  the  United  Nations  Assistance 
Mission  Afghanistan. 
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1.  Introduction 

The  Combined  Joint  Civil-Military  Operations  Task  Foree  (CJCMOTF)  eonduets  eivil 
military  operations  throughout  the  Afghanistan  Combined  Joint  Operational  Area 
(CJOA).  The  CJCMOTF  enables  seeurity,  faeilitates  reeonstruetion,  and  promotes  good 
governanee  in  the  areas  it  operates.  Additionally,  it  faeilitates  eommunieations  between 
the  Afghan  eentral  government  and  provineial,  district,  and  local  leaders.  These  efforts 
help  legitimize  the  nascent  Afghan  government  in  the  eyes  of  the  Afghan  people  and 
reduce  causes  of  instability. 

The  primary  instruments  through  which  CJCMOTF  operates  are  the  Provincial 
Reconstruction  Teams  (PRTs).  The  PRT  concept  was  developed  in  concert  with  the 
Afghan  government  and  has  its  full  support.  The  first  PRT  was  established  under  United 
States  (US)  command  in  Gardez  in  December  2002  to  serve  the  eastern  United  Nations 
Assistance  Mission  Afghanistan  (UNAMA)  region.  Today  there  are  also  Coalition  and 
NATO  PRTs  in  Bamian  (New  Zealand),  Mazar-e-Sharif  (United  Kingdom),  Konduz 
(Germany),  Parwan  (US),  Kandahar  (US),  Herat  (US),  and  Jalalabad  (US).  Four 
additional  PRTs  are  expected  to  open  by  February  2004  in  Ghazni,  Asadabad,  Khowst, 
and  Qalat,  with  a  total  of  16  PRTs  expected  to  be  operational  by  the  summer  of  2004. 

The  PRTs  are  essential  to  strengthening  the  reach  of  the  Afghan  government  and 
improving  the  ability  of  the  government  to  effectively  govern. 

We  identified  the  need  to  develop  a  system  of  metrics  to  assess  progress  in  achieving 
desired  tactical  and  operational  effects  toward  a  defined  civil-military  operations  (CMO) 
end  state.  Therefore,  we  initiated  a  program  to  develop  CMO  measures  of  effectiveness 
(MOE)  for  Afghanistan  in  July,  2003.  This  report  documents  the  process  followed  and 
the  system  developed.  Lessons-learned  are  incorporated. 

2.  Development  of  Measures  of  Effectiveness 
2,1,  General 

The  program  to  establish  MOE  for  the  CJCMOTE  CMO  in  Afghanistan  was  initiated  to 
systematically  assess  progress  in  achieving  effects  toward  a  defined  end  state.  Although 
the  specific  application  was  development  of  MOE  for  CMO,  the  process  described  below 
can  be  applied  to  any  type  of  operation.  Measures  of  effectiveness  are  an  essential 
component  of  effects-based  operations,  enabling  the  commander  to  assess  effects  as  part 
of  his  decision  cycle.  According  to  the  effects-based  operations  concept,  the  steps  in  the 
commander’s  battle  space  system,  or  decision  cycle,  include  assess,  plan,  execute,  adapt, 
and  assess,  (US  Joint  Eorces  Command,  2003).  Although  the  effects-based  operations 
approach  is  new,  the  concept  of  assessing  effects  as  an  integral  part  of  the  commander’s 
decision  cycle  is  not.  According  to  Army  doctrine,  assessing  is  an  indispensable  step  in 
the  commander’s  decision  cycle  (US  Department  of  the  Army,  2001).  Joint  doctrine  for 
CMO  specifically  addresses  the  use  of  MOE  as  a  means  of  gathering  information  for 
input  into  the  commander’s  decision  cycle  (US  Joint  Chiefs  of  Staff,  2001). 
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Assessment  requires  a  systematie  method  for  monitoring  progress  toward  attaining 
the  desired  end  state.  Often,  easily  quantifiable  metries  are  seleeted  for  use  in  the 
assessment  phase.  These  frequently  inelude  meetings  eompleted;  projeets  eompleted; 
dollars  spent;  sehools,  wells,  and  elinies  eonstrueted;  and  others.  Eaeh  of  these  represents 
a  measure  of  how  many  of  a  given  type  of  task  is  eompleted,  sueh  that,  for  example,  a 
projeet  eompleted  is  a  task  eompleted.  These  may  be  termed  measures  of  performanee 
(MOP).  Metries  sueh  as  MOP  are  readily  quantifiable  and  provide  useful  information. 
However,  monitoring  the  number  of  tasks  performed  does  not  neeessarily  provide  reliable 
information  about  progress  toward  an  end  state.  To  reliably  monitor  progress  toward  a 
defined  end  state,  MOE  must  be  used. 

Measures  of  effeetiveness  may  be  eontrasted  with  MOP,  whieh  are  mueh  more  widely 
known  and  easier  to  identify.  Measures  of  performanee  allow  quantifieation  of  the 
number  of  tasks  eompleted.  They  may  also  provide  information  on  how  effieiently  tasks 
are  performed,  for  example,  average  time  to  award  eontraets  or  the  number  of  projeets 
eompleted  on  time  may  be  monitored.  The  important  point  is  that,  in  general,  tasks  are 
assigned  to  aehieve  effeets.  Effeets  are  desired  to  reaeh  eonditions  that  eharaeterize  a 
defined  end  state.  Theoretieally,  tasks  eould  be  very  well  exeeuted  but  not  produee  the 
desired  effeets  or  provide  any  progress  toward  the  desired  end  state.  This  situation  would 
be  very  difficult  to  identify  by  monitoring  only  MOP.  By  monitoring  MOE,  specific 
information  is  obtained  on  whether  and  to  what  degree  desired  effects  are  obtained  by 
performance  of  assigned  tasks.  According  to  Joint  Doctrine  (EIS  Joint  Chiefs  of  Staff, 
2001),  MOE:  1)  assist  the  commander  in  determining  when  all  or  part  of  the  mission  has 
been  accomplished;  2)  provide  a  means  to  evaluate  the  contributions  of  military  efforts  to 
the  larger  desired  end  state;  and  3)  provide  a  baseline  of  indicators.  Erom  a  broader 
perspective,  MOE  provide  information  on  progress  toward  the  defined  end  state,  enabling 
the  most  informed  decisions  possible. 

2.2,  Characteristics  of  Measures  of  Effectiveness 

It  is  useful  to  consider  the  important  characteristics  of  effective  MOE.  In  general,  MOE 
should  (be): 

1 .  Appropriate:  provide  information  useful  to  the  commander  and  staff  or  other 
audience. 

2.  Mission-related:  provide  meaningful  assessment  of  progress  toward  the  end  state. 

3.  Measurable:  quantitative  is  preferred. 

4.  Reasonable  in  number:  too  many  becomes  unmanageable. 

5.  Timely:  timely  detection  of  effects  allows  timely  input  for  the  commander  to  respond. 

6.  Reliable:  two  measures  of  same  item  at  same  time  give  same  result. 

7.  Specific:  measures  indicator  of  interest  with  minimal  interference  from  other  factors. 
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8.  Sensitive:  can  detect  important  changes. 

9.  Feasible:  do-able  by  the  proposed  personnel. 

10.  Measured  using  a  standardized  collection  tool  or  method. 

1 1 .  Provide  information  useful  at  multiple  levels  of  command. 

2.3.  Approach  to  the  Development  of  Measures  of  Effectiveness 

To  most  effectively  develop  MOE,  a  systematic  and  prospective  approach  must  be  used. 
It  is  possible,  and  perhaps  sometimes  necessary,  to  select  MOE  as  tasks  are  defined 
during  the  course  of  a  long  term  operation.  However,  this  reactive  method  of  MOE 
identification  has  at  least  three  important  drawbacks.  The  first  is  that  this  approach  risks 
the  development  of  multiple  MOE  that  are  tied  only  to  specific  tasks  and  become 
obsolete  when  the  task  is  complete,  and  may  not  contribute  to  a  focused  and  manageable 
approach  to  MOE.  The  second  is  that  it  may  be  impossible  to  clearly  see  how  the  effects 
attained  fit  into,  and  define  progress  toward,  the  overall  end  state  and  any  related  interim 
states  along  the  pathway  to  the  final  end  state.  Third,  this  approach  deprives  staffs  of 
visibility  on  the  overall  array  of  potential  MOE  that  may  be  employed  throughout  a  series 
of  related  operations. 

The  following  steps  should  be  taken  in  developing  MOE.  In  practice  it  may  be 
necessary  to  use  an  abbreviated  approach  due  to  time  constraints,  similar  to  the  approach 
to  abbreviation  of  the  standard  military  decision  making  process.  Omission  of  any  step 
should  be  based  on  a  conscious  decision,  with  the  implications  of  omission  understood  in 
advance.  There  are  two  major  components  to  this  process,  1)  End  state  definition  and 
identification  of  potential  effects  and  2)  MOE  selection  and  refinement. 

2.4,  End  state  Definition  and  Identification  of  Potential  Effects 

1 .  Characterize/define  the  desired  end  state.  The  end  state  should  be  viewed  as  being 
made  up  of  a  set  of  conditions  that  must  be  present  when  all  operations  are  complete. 

2.  Determine  what  set  of  (potential)  effects  results  in  the  end  state.  Effects  are  closely 
related  to  conditions,  and  may  be  thought  of  as  the  changes  that  must  take  place  so  that 
the  desired  set  of  conditions  exists.  Multiple  effects  are  often  required  to  move  the 
system  (enemy,  populace,  etc.)  from  the  present  or  initial  state  (initial  set  of  conditions) 
to  the  desired  end  state. 

3.  Determine  whether  a  relevant  threshold  can  be  defined  for  each  effect  that  identifies 
when  that  effect  has  been  achieved  to  the  desired  level.  Thresholds  are  not  necessary  to 
develop  a  MOE  program  but  are  useful  when  possible  to  define. 

4.  Identify  pathways  to  arrive  at  the  desired  end  state.  Eor  example,  a  three  phase 
operation  will  have  a  pathway  involving  execution  of  the  three  successive  phases. 
Another  way  to  look  at  this  is  to  develop  courses  of  action. 
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5.  Identify  interim  states  along  eaeh  pathway,  or  the  seleeted  pathway  (eourse  of  aetion). 
Interim  states  may  be  thought  of  as  the  “end  states”  after  each  successive  operation  or 
phase  in  a  series  that  leads  along  a  pathway  to  the  “final”  desired  end  state.  For  the 
example  of  a  three  phased  operation  above,  the  end  states  after  each  of  phases  one  and 
two  would  be  “interim”  states.  The  “final”  end  state  would  exist  after  phase  three. 

Interim  states  are  not  required  but  are  very  important  to  identify  when  possible. 

6.  Determine  what  set  of  (potential)  effects  results  in  each  interim  state. 

2.5,  Measure  of  Effectiveness  Selection  and  Refinement 

For  each  effect  in  the  sets  of  effects  for  the  interim  states  and  final  end  state: 

1 .  Determine  the  pattern  of  change  associated  with  progress  toward  each  interim  state  (if 
any)  and  end  state  for  each  identified  effect. 

2.  Identify  potential  indicators  of  progress  for  each  effect. 

3.  Identify  common  indicators  that  may  cross  over  effects. 

4.  Identify  sentinel  indicators  (e.g.  if  one  indicator  is  always  indicative  of  some  other 
indicators,  it  could  be  a  sentinel  indicator  for  the  others). 

5.  Make  a  preliminary  selection  of  candidate  indicators  to  become  MOE. 

6.  Assess  each  potential  MOE  based  on  the  desired  generic  MOE  characteristics,  as 
described  above. 

7.  Select  the  initial  set  of  MOE. 

8.  Conduct  pilot  tests  of  initial  MOE  to  determine  feasibility  and  utility. 

9.  Based  on  results  of  pilot  testing,  select  final  set  of  MOE. 

10.  Remember  that  MOE  may  change  as  situation,  available  information,  mission,  etc. 
change. 

2.6,  Measures  of  Effectiveness  as  Part  of  a  System 

A  complete  set  of  MOE  will  provide  a  comprehensive  means  of  assessing  progress 
toward  all  or  nearly  all  of  the  conditions  that  make  up  the  desired  end  state.  The  set  of 
MOE  should  be  incorporated  into  a  system  designed  to  monitor  MOE.  Measures  of 
effectiveness  are  useful  in  and  of  themselves  in  assessing  effects  for  discrete  operations 
of  limited  scope.  They  may  be  used  individually  in  a  limited  way  but  will  be  most 
effective  when  incorporated  into  a  comprehensive  system. 
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Several  points  should  be  considered  in  developing  a  MOE  system.  The  system 
includes:  1)  measurement,  which  includes  the  measurement  method  or  tool  and  the 
people  performing  the  measurement,  and  which  produces  data;  2)  data  communication, 
which  includes  communication  of  data  obtained  during  measurement  to  a  central  data 
collection  point;  3)  data  entry  and  storage,  which  includes  entry  of  data  into  a  data  set  or 
data  base;  4)  data  handling  and  analysis;  and  5)  results  interpretation  and  reporting.  The 
measurement  component  is  addressed  in  the  ideal  characteristics  of  MOE  above.  The 
data  communication  component  may  be  different  for  different  MOE  within  the  system. 
The  key  is  that  all  necessary  communications  pathways  must  be  clearly  defined. 

Methods  to  communicate  data  should  include  measures  to  protect  against  transcription 
errors,  data  corruption,  and  data  loss,  and  should  minimize  required  effort.  Data  entry  is 
a  point  where  transcription  error  may  be  common.  Either  a  data  set  or  a  data  base  format 
may  be  used  for  data  storage,  based  on  the  needs  of  the  system,  available  expertise,  and 
preference.  The  results  and  interpretation  component  should  be  given  significant  thought. 
This  is  a  point  where  specific  expertise  is  necessary.  Most  if  not  all  of  the  data  collected 
as  MOE  will  require  statistical  analysis.  Statistical  analysis  is  the  only  means  that  allows 
determination  of  a  change  (effect)  with  a  specified  degree  of  confidence  that  the  change 
is  real,  and  not  just  due  to  random  chance.  In  some  cases,  apparent  effects  may  be  so 
numerically  large  that  they  appear  to  obviously  represent  true  changes.  However,  even 
apparently  large  effects  may  actually  be  due  to  random  chance.  There  is  no  substitute  for 
proper  statistical  analysis.  That  having  been  said,  data  analysis  for  MOE  can  be 
accomplished  using  simple  statistics.  More  sophisticated  analyses  may  be  necessary  and 
desirable.  Consider  using  reach-back  capability  (specialized  capability  attained  through 
cooperation  with  agencies  located  in  the  United  States  or  elsewhere  outside  of  theater)  for 
these  analyses  if  the  proper  statistical  expertise  is  not  present  or  available  within  the  unit 
or  with  an  interagency  partner  in  theater. 

Before  developing  a  system,  it  is  important  to  conduct  a  pilot(s)  using  MOE, 
including  data  handling  and  analysis.  This  will  allow  early  elimination  of  unsuitable 
MOE  and  will  give  a  realistic  preview  of  the  levels  of  effort,  automation, 
communications,  and  technical  expertise  that  will  be  necessary.  Understand  that  the 
system  will  evolve  as  improvements  become  apparent  or  mission  requirements  change.  It 
is  important  to  note  that,  while  mission  requirements  or  the  general  approach  to 
operations  (e.g.  lines  of  operation)  may  change,  the  desired  end  state  is  not  likely  to 
change  much  if  at  all  during  a  campaign.  Because  the  MOE  and  the  system  as  a  whole 
are  based  on  the  end  state,  as  opposed  to  a  given  approach  (e.g.  lines  of  operation),  the 
system  will  be  able  to  adapt  to  changes  with  minimal  impact.  The  required  MOE  should 
drive  the  design  of  the  system.  Developing  a  system  for  another  purpose  and  then 
selecting  only  MOE  that  are  compatible  with  the  existing  software  system  severely  limits 
flexibility  in  MOE  selection  and  should  be  avoided.  Once  a  flexible  MOE  system  that 
uses  commonly  available  software  is  developed,  it  can  be  adapted  to  future  MOE. 
Elexibility  is  important.  Specifically  designed  software  systems  are  not  necessary  and  are 
likely  to  be  difficult  to  use  and  adapt. 

Properly  designed  and  implemented  MOE  provide  critical  information  on  whether 
and  to  what  degree  operations  have  the  intended  effects.  Assessment  of  effects  is  an 
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integral  part  of  the  commander’s  decision  cycle,  regardless  of  the  operational  paradigm 
employed.  When  considered  along  with  other  sources  of  information,  such  as 
intelligence  and  information  from  the  other  battlefield  operating  systems,  MOE  enables 
the  most  effective  decision  making.  A  distinction  has  been  made  between  MOE  and 
MOP,  with  the  importance  of  MOE  emphasized.  In  most  cases,  commanders  will  want 
some  information  on  MOP  as  well  as  MOE.  Measures  of  performance  can  be  very 
useful,  as  long  as  the  distinction  between  MOE  and  MOP  is  clear  and  confusion  between 
the  two  avoided.  Measures  of  effectiveness  are  used  to  determine  whether  or  not  the 
tasks  performed  were  effective.  The  bottom  line  is  that  the  MOE  system  must  yield 
information  that  is  useful  for  the  commander  in  decision  making,  when  considered  in 
light  of  all  other  available  information. 
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3,  Civil  Military  Operations  End  State  Definition 

3.1.  Previously  Defined  End  States 

The  descriptions  of  end  states  presented  here  are  based  on  information  that  was  current  at 
the  time  that  the  CMO  end  state  definition  process  was  executed.  It  is  understood  that 
this  information  is  subject  to  change. 

Key  documents  describe  the  Afghan  end  state  from  different  perspectives.  Each 
perspective  is  important  to  consider  in  developing  the  CMO  end  state  for  Afghanistan. 
Foremost  is  the  Afghan  end  state  set  forth  in  President  Karzai’s  National  Program  for 
Reconstruction  (ITSA,  2002).  The  Government  of  Afghanistan’s  (GoA)  vision  for  the 
ultimate  Afghan  end  state  is  for  a  “prosperous,  secure  Afghanistan;  a  nation  built  on  a 
credible  State  with  a  transparent  and  accountable  administration,  a  well-developed  civil 
society  with  democratic  institutions  supported  and  being  supported  by  the  rule  of  law.  In 
this  environment  an  effective  and  competitive  private  sector  will  flourish  allowing  the 
people  of  Afghanistan  to  realize  the  potential  of  their  own  natural  and  human  resources.” 
The  National  Development  Framework  for  Afghanistan  was  devised  as  a  road  map  to 
reach  this  end  state.  The  National  Development  Framework  involves  12  national 
programs:  1)  Refugee  Return;  2)  Education;  3)  Health  and  Nutrition;  4)  Fivelihoods;  5) 
Culture;  6)  Transport;  7)  Urban  Management;  8)  Energy,  Mining  and 
Telecommunications;  9)  Natural  Resource  Management;  10)  Trade  and  Private 
Investment;  11)  Public  Administration;  and  12)  Security  and  Rule  of  Faw.  It  is  clear  that 
reaching  this  ultimate  Afghan  end  state  will  be  a  long  process  involving  many 
organizations,  each  with  one  or  more  roles  to  play. 

3.1.1.  The  document  commonly  referred  to  as  the  Bonn  Agreement  (UNDP,  2002)  sets 
forth  the  schedule,  steps  to  be  taken,  and  the  United  Nations’  Role  in  establishing  a 
democratic  government  in  Afghanistan.  Major  milestones  in  this  process  include 
establishment  of  the  Islamic  Transitional  Government  of  Afghanistan  (ITGA),  the 
drafting  of  a  new  constitution  by  the  Constitutional  Foya  Jirga,  and  national  elections. 
The  Bonn  Agreement  is  a  principal  guiding  document  for  all  involved  in  reconstruction 
and  security  efforts  in  Afghanistan,  particularly  our  multilateral  partners,  the  United 
Nations  Assistance  Mission  Afghanistan  (UNAMA)  and  the  International  Security 
Assistance  Force  (ISAF). 

3.1.2.  The  United  States  Department  of  State  (DoS)  has  described  a  shorter  term 
perspective  of  the  Afghan  end  state  (US  DoS,  2002).  The  DoS  desired  end  state  for 
Afghanistan  after  two  years  is  described  in  terms  of  Political,  Security,  Economic,  and 
Humanitarian  conditions. 

3 . 1 .2. 1 .  Political  conditions  at  the  end  of  two  years  include: 

1 .  Successful  conclusion  to  the  Bonn  process. 

2.  A  new  constitution  in  place  ratified  by  the  Constitutional  Foya  Jirga. 
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3.  A  balance  of  regional  autonomy  and  central  authority  mindful  of  Afghanistan’s 
history. 

4.  A  broad-based  eentral  authority  having  started  to  provide  basic  social  services  to  its 
25  million  people. 

5.  Political  decision-making  on  the  basis  of  representative  majority  rule  and  eonsensus 
as  opposed  to  warlordism  and  force  of  arms. 

3. 1.2. 2.  In  terms  of  seeurity,  DoS  end  state  eonditions  include: 

1 .  Afghanistan  no  longer  a  safe  harbor  for  terrorists  or  their  finaneial  and  material 
resources. 

2.  Sufficient  internal  stability  for  delivery  of  aid  and  development  assistanee. 

3.  “Spoilers”  neutralized  or  marginalized  (“Spoilers”  are  individuals  and/or  entities 
working  against  the  legitimizing  of  the  Afghan  Government). 

4.  National  Military  and  Police  assuming  responsibility  for  internal  seeurity,  with 
representation  from  major  ethnie  and  regional  groups  and  modern  eommand 
struetures;  respectful  of  human  rights.  Continued  need  for  external  monitoring. 

5.  Relative  peaee  and  eooperation  among  the  eountry’s  regions  and  with  its  neighbors. 

3. 1.2. 3.  Eeonomie  eonditions  inelude: 

1 .  Eeonomy  re-emerging  through  agrieultural  development,  restored  infrastrueture, 
inereased  international  trade  and  small-seale  industry. 

2.  Eeonomie  struetures  whieh  respeet  Islamic  Eaw  and  permit  normal  international 
eeonomie  interaction  and  development. 

3.  Sehools,  ineluding  voeational  sehools,  raising  the  quality  of  the  workforee. 

4.  Viable  alternatives  emerging  to  banditry  and  nareo-traffieking. 

5.  Re-engagement  in  regional  eeonomie  and  trading  relationships. 

3 . 1 .2.4.  Humanitarian  eonditions  include: 

1 .  Moving  towards  agrieultural  self-sufficiency. 

2.  Reconstruction  and  development  efforts  well  underway. 

3.  Refugees  returning  home. 
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4.  Dispute  resolution  mechanisms  in  place. 

5.  De-mining  operations  well  underway. 

3.1.3.  According  to  the  US  Combined  Joint  Chiefs  of  Staff,  the  desired  end  state  is  a 
Go  A  that  is: 

1 .  Moderate  and  democratic,  though  understanding  that  Afghans  will  not  copy  US-style 
institutions. 

2.  Representative  of  all  responsible  elements  in  Afghanistan  and  formed  through  the 
political  participation  of  the  Afghan  people. 

3.  Capable  of  effectively  controlling  and  governing  its  territory. 

4.  Capable  of  implementing  policies  to  stimulate  economic  development. 

5.  Willing  to  contribute  to  a  continuing  partnership  with  the  US-led  coalition  in  the 
Global  War  on  Terror. 

3.1.4.  The  United  States  Central  Command  (USCENTCOM)/Combined  Forces 
Command-Central  Asia  (CFC-CA)  end  state  includes  the  following: 

1 .  The  GOA,  unilaterally  or  with  the  assistance  of  an  established  international  security 
force,  capable  of  preventing  the  re-emergence  of  terrorism  in  Afghanistan. 

2.  Classified. 

3.  Classified. 

4.  Classified. 

5.  Classified. 

6.  Classified. 

3.1.5.  The  end  state  set  forth  by  Combined  Joint  Task  Force-180  (CJTF-180)  is:  GoA 
committed  to  and  capable  of  preventing  the  re-emergence  of  terrorism  on  Afghan  soil. 

3.2,  Civil-Military  Operations  End  State  Development 

These  end  states  are  compatible  in  that  each  end  state  can  be  viewed  as  an  interim  state 
along  the  pathway  to  the  ultimate  Afghan  end  state,  as  set  forth  by  the  GoA.  If  one 
considers  the  pathway  to  the  ultimate  Afghan  end  state  as  a  timeline  beginning  with  the 
time  of  Taliban  rule  and  ending  with  the  ultimate  Afghan  end  state,  the  pathway 
progresses  through  intermediate,  or  interim,  states  (Figure  1).  In  general,  each  of  the 


9 


USACAPOC  Technical  Report 


many  organizations  involved  in  the  proeess  of  rebuilding  Afghanistan  has  a  diserete  role 
to  play,  with  a  eorresponding  segment  of  the  timeline.  Segments  of  various  organizations 
may  overlap  and  individual  organizations  may  have  multiple  roles  with  multiple 
eorresponding  segments  of  the  timeline.  Eaeh  of  the  end  states  listed  above  has  a  plaee 
along  this  timeline.  Seleeted  key  milestones  or  end  states  are  listed  along  the  timeline  in 
Figure  1  for  illustration. 

Although  end  states  for  Afghanistan  were  previously  identified  by  the  US  DoS,  US 
CJCS,  USCENTCOM,  and  CJTF-180,  as  well  the  GoA,  CJCMOTF  required  a  greater 
degree  of  end  state  definition  to  enable  the  development  of  a  reliable  set  of  MOEs.  The 
first  step  was  to  define  the  eivil  military  end  state  as  the  set  of  eonditions  and  eapabilities 
of  the  Afghan  government  that  must  exist  sueh  that  1)  the  major  eoalition  military 
presenee  ean  depart  Afghanistan  and  2)  the  Afghan  government  will  eontinue  to  progress 
with  only  the  assistanee  of  nongovernmental  organizations  (NGO),  eivilian  eoalition 
government  representatives,  and  eontinued  international  fiseal  and  material  aid.  It  was 
further  reeognized  that  this  CMO  end  state  was  in  effeet  an  interim  state  on  the  pathway 
to  the  ultimate  Afghan  end  state  as  deseribed  by  the  GoA  (Figure  1). 

As  part  of  determining  the  set  of  eonditions  and  eapabilities  that  eomprise  the  CMO 
end  state,  a  one-day  workshop  was  eondueted  in  Kabul.  Attendees  ineluded:  seleeted 
personnel  from  CJCMOTF,  CJTF-180,  the  Offiee  of  Military  Cooperation-Afghanistan 
(OMC-A),  US  DoS,  and  UNAMA.  The  US  Ageney  for  International  Development 
(USAID)  was  invited  but  eould  not  attend  and  provided  input  separately.  Partieipants 
were  asked  to  1)  define  the  essential  Afghan  government  funetions  that  must  exist  to 
reaeh  the  CMO  end  state  and  2)  rank  the  importanee  of  eaeh  Afghan  ministry  relative  to 
the  end  state.  Summarized  results  are  shown  in  tables  1  and  2. 

Based  on  results  of  the  workshop  and  the  produets  of  prior  mission  analysis,  a 
working  group  further  refined  the  CMO  end  state  definition,  whieh  may  be  summarized 
as:  A  Legitimate  Afghan  Government  that  demonstrates  1)  Competency;  2)  Influence;  3) 
Democracy  that  is  Sustainable,  Participatory,  and  Rights-respecting,  and  4)  Core 
Deliverables  in  the  Areas  of  Security,  Justice,  Education,  and  Mechanisms  for  Economic 
Development.  Taken  together,  the  first  three  eomprise  the  fundamentals  of  good 
governanee,  whieh  are  important  for  all  funetions  of  government.  The  fourth  identifies 
government  funetions  that  must  be  established  to  varying  degrees  to  aehieve  the  CMO 
end  state.  Further  definition  of  the  end  state  was  made  in  terms  of  eapabilities  that  the 
GoA  must  possess,  and  funetions  that  it  must  perform  to  aehieve  the  end  state.  Table  3 
lists  the  end  state  eharaeteristies,  as  listed  in  the  summarized  end  state,  along  with 
eorresponding  minimum  essential  eore  funetions,  and  eritieal  primary  funetions. 
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Figure  1.  Pathway  to  Afghan  End  State 
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Table  1.  Essential  Functions  of  the  Afghan  State  Required  to  Attain  the  CMO  End  State 


Security  (Internal  and  External) 

Rule  of  Law  (Including  Judicial  and  Enforcement) 

Economic  Development 

Education 

Eiscal  and  Monetary 

Communications 

Govern/Lead 

Table  2.  Afghan  Ministries  that  are  Essential  in  Terms  of  Attaining  the  CMO  End  State 

Defense 

Office  of  the  President 

Interior  (with  essential  Erontiers  function) 

Einance 

Justice 

Education 

Health 

Agriculture 

Communication 

Commerce 

Eoreign  Affairs 

Water  and  Power 

Public  Works 

Rural  Rehabilitation  and  Development 

Civil  Aviation  and  Tourism 
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Table  3.  Civil  Military  End  State  Government  Characteristics  and  Functions 


End  State 
Characteristics 

Minimum  Essential  Core 
Functions 

Critical  Primary  Functions 

Competency 

Conduct  Government  Business 
Effectively,  Efficiently, 
Responsively,  and  in  an 
Accountable  Manner 

Communicate 

Coordinate  Aid  Efforts 

Establish  and  Enforce  Quality 
Standards  for  Government  Goods 
and  Services  (Including  Aid 
Projects) 

Establish  Bureaucracy 

Be  Responsive  to  the  Needs  of  the 
People 

Conduct  Strategic  Planning 

Respect  Human  Rights 

Inform  the  Public  of  Government 
Activities 

Conduct  Personnel  Management 

Provide  Physical  Infrastructure  for 
Government  Eunction 

Cooperate  with  Other  Parts  of 
Government 

Disseminate  Information 

Plan  and  Execute  Budgets 

Disburse  Eunds 

Influence 

Generate  Support  of  the  Populace 
for  GoA  and  Key  Institutions 

Generate  Support  of  Population 
for  ITGA 

Generate  Support  of  Population 
for  the  Afghan  National  Army 
(ANA) 

Generate  Support  of  Population 
for  the  Police 
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Table  3.  Civil  Military  End  State  Government  Characteristics  and  Functions  (continued) 


End  State 
Characteristics 

Minimum  Essential  Core 
Functions 

Critical  Primary  Functions 

Democracy 

Execute  Democratic  Processes  that 
are  Participatory,  Consensus 
Oriented,  Equitable,  Inclusive,  and 
Sustainable 

Ratify  the  Constitution 

Conduct  National  Elections  that 
have  National  and  International 
Eegitimacy 

Core  Deliverables 

Provide  Security 

Establish  National  Security 
Apparatus 

Eully  establish/reform  the  Ministry 
of  Defense 

Train  and  Staff  ANA  at  Target 
Eevels 

Establish  National  Police 

Establish  Regional  Police 

Staff  Eocal  and  Regional  Police  at 
Proper  Eevels  (Based  on 

Population  by  District) 

Establish  Eunctional,  Properly 
Equipped,  and  Properly  Staffed 
Police  Stations  in  each  District 

Establish  and  Staff  Highway 

Patrol 

Control  Borders 

Complete  the  Disarmament, 
Demobilization,  and  Reintegration 
(DDR)  process 

Provide  Security 

Provide  Intelligence  (Internal  and 
External) 

Conduct  International  Diplomacy 

Provide  Justice 

Establish  National  Justice  System 
and  Procedures 

Establish  a  Eunctioning  Eederal 
Court  System 

Provide  Eunctioning  Jails  that 

Meet  International  Minimum 
Standards  in  Each  Province 
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Table  3.  Civil  Military  End  State  Government  Characteristics  and  Functions  (continued) 


End  State 
Characteristics 

Minimum  Essential  Core 
Functions 

Critical  Primary  Functions 

Core  Deliverables 
(continued) 

Make  Quality  Education  Available 
to  Citizens 

Establish  National  System  and 
Standards  for  Education 

Make  Elementary  Education 
Available  and  Compulsory  for 
Children  in  Ah  Districts 

Establish  Plans  for  Making 
Secondary  Education  Widely 
Available 

Make  Eiteracy  Training  Available 
for  Adults  in  ah  Districts 

Establish  Basic  Mechanisms  for 
Economic  Development 

Collect  Revenue  Effectively  and 
Systematically 

Establish  a  Sustainable 

Agricultural  Development 

Program  to  Develop  Agricultural 
Exports  and  Agricultural  Self- 
sufficiency 

Build  Road  Infrastructure  to 

Connect  Major  Cities  and  Plan 

Road  Construction  to  Connect 
Agricultural  Producers  to  Markets 

Establish  Policies  to  Promote 
Economic  Development 
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4,  Overall  Measures  of  Effectiveness  System 

4.1,  General 

Three  major  eomponents  were  identified  for  inelusion  in  the  overall  MOE  system  to 
address  all  important  funetions  identified  in  the  CMO  end  state:  1)  assessment  of 
governanee  quality  indieators  (AGQI);  2)  assessment  of  governanee  meehanisms  (AGM); 
and  3)  national  key  program  indieators. 

4.2,  Assessment  of  Governance  Quality  Indicators 

The  AGQI  is  an  assessment  of  the  quality  of  governanee  from  the  point  of  view  of 
eonsumers  of  government.  It  takes  the  form  of  an  interview  and  is  primarily  eondueted  at 
the  level  of  average  eitizens,  business  people,  and  low-level  government  and  traditional 
leaders.  However,  the  AGQI  may  be  used  with  any  eategory  of  person.  The  data 
produeed  are  of  signifieant  use  in  the  assessment  phase  of  the  eommander’s  deeision 
eycle  at  the  PRT  level,  and  also  when  eonsolidated  at  higher  levels.  Additionally,  the 
data  are  useful  for  multilateral  deeision  makers.  The  AGQI  is  diseussed  in  detail  in 
another  seetion  of  this  report. 

4.3,  Assessment  of  Governance  Mechanisms 

The  AGM  is  an  assessment  performed  at  the  eity/distriet  to  ministerial  levels  of 
government  to  examine  the  meehanisms  employed  by  government  in  the  following 
eategories:  1)  Communieations;  2)  Aid  Coordination;  3)  Standards;  4)  Bureaueraey;  5) 
Responsiveness  to  Needs  of  the  People;  6)  Strategie  Planning;  7)  Human  Rights;  8) 
Informing  the  Publie;  9)  Personnel  Management;  10)  Cooperating  with  Other  Parts  of 
Government;  11)  Physieal  Infrastrueture;  and  12)  Budget.  The  AGM  requires  eomplete 
eooperation  of  the  government  offiee  being  assessed.  It  entails  an  in-depth  assessment  by 
personnel  with  speeialized  skills  in  various  funetions  of  government.  Eaeh  AGM  may 
take  0.5  to  2.0  days.  The  data  produeed  will  be  valuable  at  the  level  of  the  PRT  when 
used  in  a  mentorship  relationship  with  the  government  leader(s).  When  eompiled  aeross 
regions,  the  data  will  be  useful  in  the  assessment  phase  of  the  eommander’s  deeision 
eyele  for  operations  at  the  operational  (CJCMOTE)  level.  The  data  will  also  be  useful  for 
multilateral  deeision  makers.  The  AGM  has  not  yet  been  fully  developed  and  will  be 
detailed  in  a  separate  report. 

4.4,  National  Key  Program  Indicators 

Data  on  certain  minimum  essential  functions  of  government  identified  in  the  CMO  end 
state  are  monitored  by  organizations  outside  of  the  CJCMOTE,  either  in  the  form  of  MOE 
for  those  organizations,  or  as  a  matter  of  routine.  National  Key  Program  Indicators 
provide  effectiveness  data  on  national  level  programs  or  activities  critical  to  CMO  end 
state.  The  following  is  a  list  of  these  national  level  programs  and  activities,  and  the 
organizations  or  nations  that  have  the  lead  for  monitoring  or  implementation. 

1 .  Security  status  (UNAMA;  CJTE-l  80  CJ2) 

2.  Afghan  National  Army  status  (OMC-A) 
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3.  Police  status  (Germany;  US  DoS) 

4.  Judicial  system  status  (Italy) 

5.  Demoeracy  (CLJ,  Elections;  UNAMA) 

6.  Disarmament  Demobilization  and  Reintegration  (DDR)  program  (UNAMA;  Japan) 

7.  Infrastrueture  (USAID;  multiple  international  donors) 

8.  Potentially  others  based  on  the  requirements  of  interagency  partners 

4.5,  Tactical  Level  Versus  Operational  Level  MOE 

The  AGQI  can  be  viewed  as  tactieal  level  MOE,  while  the  AGM  may  be  viewed  as 
operational  level  MOE.  In  this  theater,  CJCMOTE  operations  occur  at  the  operational 
level,  while  PRT  operations  take  plaee  at  the  tactieal  level.  The  terms  AGQI  and  AGM 
are  more  deseriptive  and  more  broadly  applieable  in  the  interagency  and  multilateral 
environment.  Therefore,  the  terms  AGQI  and  AGM  will  be  used  throughout  this  report. 

4.6,  Flow  of  Information  and  Reports 

The  planned  flow  of  information  in  the  CMO  MOE  system  is  depleted  in  figure  2.  Both 
AGQI  and  AGM  will  be  condueted  by  civil  affairs  teams  and  interageney  personnel 
working  out  of  PRTs.  Tactieal  psychological  operations  teams  will  also  perform  AGQI 
when  working  at  PRTs.  The  data  produced  will  be  used  directly  by  the  PRT  and  also 
forwarded  to  CJCMOTE  for  compilation  and  analysis.  It  is  expected  that  AGQI  and 
possibly  AGM  will  also  be  performed  by  UNAMA  personnel  who  are  not  working  out  of 
a  PRT.  In  that  situation  UNAMA  will  submit  data  directly  to  CJCMOTE  (not  depicted  in 
the  figure).  National  Key  Program  Indieators  data  will  be  reported  directly  by  the 
responsible  agency  to  CJCMOTE.  The  CJCMOTE  will  compile  and  analyze  data  and 
produce  reports  of  results  that  will  be  used  locally,  sent  to  PRTs,  and  sent  to  all  involved 
multilateral  partners.  In  addition,  periodic  reports  will  be  submitted  to  higher  levels  of 
command  and,  potentially,  to  Congress. 

In  the  form  presented  here,  the  CJCMOTE  acts  as  the  central  point  for  data 
compilation,  analysis,  and  reporting  for  the  CJOA.  As  eommand  relationships, 
responsibilities,  and  mission  requirements  change,  the  system  can  be  easily  adapted.  The 
primary  modification  would  be  to  ehange  the  central  point  for  data  analysis.  Eor 
example,  if  this  funetion  were  to  move  to  the  level  of  CEC-CA,  the  principal  change 
would  be  depicted  in  figure  2  by  substituting  CEC-CA  for  CJCMOTE.  Of  course,  some 
reporting  relationships  would  also  change.  It  is  also  conceivable  that  at  some  point  the 
primary  responsibility  for  data  analysis  and  reporting  may  move  to  an  interageney  partner 
such  as  USAID.  The  system  has  been  designed  to  be  flexible  to  allow  changes  with 
minimal  impact  on  operations  and  reporting  relationships. 


17 


USACAPOC  Technical  Report 


4,7,  Responsibilities  for  Specialized  Functions 

Some  specialized  functions  are  required  within  the  system  that  are  not  readily  available  or 
available  in  sufficient  quantity  within  the  military  or  interagency  staffs  assigned  to  the 
CJOA.  Functions  such  as  data  compilation,  statistical  analysis,  and  report  preparation 
can  be  accomplished  by  contract  or  other  personnel  either  in  theater  or,  more  likely,  in  the 
United  States  or  Europe.  Such  a  reach-back  approach  will  be  effective.  This  will  only  be 
possible  if  standardized  electronic  mechanisms  for  data  reporting  are  in  place. 
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Figure  2.  Flow  of  Information  in  the  Measures  of  Effeetiveness  System 
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5.  Assessment  of  Governance  Quality  Indicators  (AGQI) 

5.1.  Development 

The  aim  was  to  develop  a  standardized  AGQI  data  colleetion  sheet  addressing  relevant 
subfunetions  derived  from  the  CMO  end  state  definition.  The  first  step  was  development 
by  a  small  working  group  of  an  initial  prototype  set  of  MOEs.  Next,  the  initial  prototype 
was  taken  to  the  Bamian  PRT.  There  it  was  reviewed  and  modified  by  a  small  working 
group  that  included  the  PRT  Commander,  US  DoS  representative,  and  experienced  civil 
affairs  team  leaders,  as  well  as  a  representative  from  CJCMOTF  headquarters.  The  data 
collection  sheet  (version  one)  was  used  in  a  pilot  at  the  Bamian  PRT.  Based  on  results  of 
the  Bamian  pilot,  version  two  of  the  data  collection  sheet  was  developed  and  sent  to  all 
PRTs  for  feedback.  The  Kandahar  PRT  used  version  two  in  a  pilot.  Version  three  is 
considered  the  final  version  (Appendix  A).  It  takes  into  account  all  suggestions  for 
improvement  and  has  undergone  additional  working  group  review  and  review  by  USAID 
and  UNAMA.  The  AGQI  data  collection  sheet  will  be  modified  to  meet  changing 
mission  requirements  in  the  future  as  needed.  It  is  important  that  major  changes  to  the 
data  collection  sheet  not  be  made  in  the  course  of  an  operation  or  campaign  because  this 
could  greatly  reduce  the  ability  to  track  changes  over  time.  Major  changes  are  not  likely 
to  be  necessary  because  the  AGQI  was  designed  based  on  a  well  defined  end  state,  and  a 
major  change  in  the  desired  end  state  in  the  course  of  a  campaign  is  not  likely.  It  is 
anticipated  that  the  AGQI  data  collection  sheet  will  be  automated  and  incorporated  into  a 
hand-held  device  to  facilitate  data  collection  and  transfer. 

One  of  the  findings  of  the  pilot  studies  was  that  the  Afghanistan  Information 
Management  System  (AIMS)  village  assessment  provides  little  useful  information  and  no 
important  information  that  is  not  acquired  through  use  of  the  AGQI  data  collection  sheet. 
Therefore,  it  has  been  decided  that  the  AIMS  village  assessment  form  will  be  used  only 
on  an  as  needed  basis,  while  the  AGQI  data  collection  sheet  will  be  completed  for  all 
interviews  and  meetings  with  local  populace  or  leaders. 

5.2,  Description 

The  AGQI  is  an  assessment  of  the  quality  of  governance  from  the  point  of  view  of 
consumers  of  government.  It  takes  the  form  of  an  interview  and  is  primarily  conducted  at 
the  level  of  average  citizens,  business  people,  and  low-level  government  and  traditional 
leaders.  However,  the  AGQI  may  be  used  with  any  category  of  person.  The  design  is 
consistent  with  accepted  procedures  for  social  science  research  of  this  type  (Bryman, 
1984;  Sechrist  and  Sidana,  1995;  Hentschel,  1998)  and  with  interview  procedures  used  to 
assess  components  of  governance  in  other  developing  countries  (World  Bank 
International,  1999a;  1999b;  1999c).  The  composition  of  the  AGQI  data  collection  sheet 
takes  into  account  previously  published  indicators  of  governance  quality  (Center  for 
Democracy  and  Governance,  1998).  The  data  produced  can  be  defined  primarily  as 
quantitative  in  nature  but  the  design  includes  information  that  would  also  allow  some 
degree  of  qualitative  interpretation  (Bryman,  1984;  Sechrist  and  Sidana,  1995;  Hentschel, 
1998).  The  process  for  collecting  the  data  (described  later)  adds  a  significant  qualitative 
component  (Bryman,  1984;  Sechrist  and  Sidana,  1995;  Hentschel,  1998).  AGQI  data  are 
of  significant  use  in  the  assessment  phase  of  the  commander’s  decision  cycle  at  the  PRT 
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level  and  also  when  eonsolidated  at  higher  levels.  Additionally,  the  data  are  useful  for 
multilateral  deeision  makers.  Although  the  AGQI  was  speeifically  designed  for  use  in 
Afghanistan,  it  will  be  useful  for  CMO  and  civilian  relief  operations  in  other  parts  of  the 
world,  with  slight  modification. 

5.3,  Training  and  Personnel 

Data  can  be  collected  rapidly,  with  each  interview  taking  approximately  30  to  40  minutes. 
The  skills  required  to  perform  the  interviews  are  common  to  civil  affairs  personnel, 
psychological  operations  personnel,  selected  categories  of  civilian  interagency  personnel 
(e.g.  USAID  personnel),  and  civilian  relief  community  personnel.  A  period  of  training 
and  rehearsal  is  required  for  all  personnel  prior  to  performing  AGQI  (Appendix  B).  In 
addition  to  the  training  included  in  Appendix  B,  it  is  important  that  rehearsals  and  after 
action  reviews  be  used  by  any  new  assessment  team  or  whenever  a  new  member  joins  a 
team. 

Although  the  AGQI  is  not  difficult  to  complete,  the  importance  of  employing  skilled 
and  properly  trained  personnel  to  perform  the  AGQI  deserves  emphasis.  The  AGQI  is 
not  analogous  to  “person-on-the-street”  opinion  polls.  Properly  trained  and  locally 
operating  personnel  should  perform  the  AGQI.  These  personnel  understand  the  potential 
effects  of  their  presence  on  responses  and  can  take  measures  to  minimize  those  effects. 
They  are  skilled  at  building  relationships  with  the  populace.  They  understand  the 
implications  of  potential  responses  and  can  lead  the  conversation  to  get  more  information 
when  warranted.  Locally  operating  personnel  understand  the  nuances  of  the  local  culture 
and  the  local  situation.  The  use  of  properly  trained  and  locally  operating  personnel  also 
makes  possible  the  high  payoff  that  comes  with  building  individual  and  organizational 
relationships  over  time. 

The  importance  of  interpreters  cannot  be  over-emphasized.  When  one  considers  that 
each  question  asked  and  each  response  given  passes  through  the  interpreter,  the 
importance  of  the  interpreter  becomes  clear.  The  interpreter  is  an  essential  part  of  the 
team.  Interpreters  should  be  treated  as  a  true  part  of  the  team,  they  should  train  and 
practice  as  part  of  the  team,  and  they  should  be  made  to  feel  that  they  are  a  part  of  the 
team.  Interpreters  must  fully  understand  the  assessment  process  and  each  question  asked 
so  that  they  can  best  relate  the  attitudes  of  the  interviewees  and  their  answers  to 
questions.  To  facilitate  complete  understanding  of  each  question  by  each  interpreter,  the 
AGQI  data  collection  sheet  should  be  translated  into  Dari  and  Pashto  to  enable  each  Dari 
or  Pashto  speaking  interpreter  to  see  the  questions  in  his  native  language.  Translations 
are  in  progress. 

5.4,  Format 

The  AGQI  data  collection  sheet  is  a  standardized  form  with  some  sections  that  are 
completed  for  all  interviews  and  other  sections  that  are  completed  only  for  leaders  or 
business  people  (Appendix  A).  The  data  collection  sheet  is  designed  to  maximize  check- 
the-block  format  to  minimize  required  writing  on  the  part  of  the  recorder.  Most  of  the 
form  is  formatted  for  answers  on  a  0  to  5  scale,  with  1  indicating  strong  disagreement,  5 
indicating  strong  agreement,  and  0  denoting  a  response  of  non-applicable  or  does  not 
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know.  The  data  collection  sheet  is  divided  into  eight  parts:  Part  1:  Demographics  (15-20 
questions);  Part  2:  Attitudes  (8  questions);  Part  3:  Security  (7  questions);  Part  4:  Human 
Rights  (2  questions);  Part  5:  Democracy  (5  questions);  Part  6:  Government  Competency 
in  Meeting  People’s  Needs  (1 1  questions);  Part  7:  Leaders  Only  (8  questions);  and  Part  8: 
Business  People  Only  (15).  Often,  it  is  possible  and  advisable  to  complete  part  of  the 
demographics  section  prior  to  the  interview.  In  total,  48  questions  are  completed  for  an 
average  citizen,  6 1  questions  are  completed  for  a  leader,  and  63  questions  are  completed 
for  a  business  person. 

5,5,  Interview  Methodology 

5.5.1.  Team  Composition.  Experience  has  revealed  some  interview  techniques  worth 
noting.  Force  protection  must  be  the  utmost  priority  and  must  be  planned  and  rehearsed. 
The  force  protection  element  should  be  separate  from  the  interview  team(s)  to  allow 
interview  teams  to  focus  completely  on  their  work.  Each  interview  team  includes  an 
interviewer,  a  recorder,  and  an  interpreter.  When  necessary,  the  interviewer  can  serve  as 
the  recorder  but  this  detracts  from  the  interviewer’s  ability  to  focus  on  the  interview.  It  is 
important  that  the  interviewer  run  the  interview.  Neither  the  interpreter  nor  the  recorder 
should  assume  the  initiative  in  this  regard.  In  practice,  rank  and  gender  of  the  interviewer 
and  recorder  have  not  been  issues.  Well  trained  personnel  in  the  rank  of  E-4  can 
successfully  perform  the  AGQI  interview,  although  noncommissioned  officers  and 
officers  should  be  considered  the  primary  interviewers.  Interagency  or  multilateral 
personnel  with  similar  levels  of  experience  may  also  be  considered  primary  interviewers. 

5.5.2.  Conducting  the  Interview.  There  are  two  basic  methods  to  locate  a  person  to 
interview.  One  is  for  the  interpreter  to  go  ahead  and  find  someone  who  is  willing  to  be 
interviewed.  Then  the  full  team  approaches  or  the  potential  interviewee  is  lead  to  the 
team.  The  other  is  for  the  entire  team  to  accompany  the  interpreter  as  he  approaches 
potential  interviewees.  Both  of  these  methods  have  applications.  Before  starting  the 
interview,  find  a  semi-private  or  private  place  to  conduct  the  interview.  The  interview 
should  be  conducted  with  only  a  single  interviewee  or  with  a  very  small  group  of  people 
with  whom  the  interviewee  is  obviously  comfortable.  The  interviewee  should  be  in  a 
position  to  be  as  candid  as  possible.  If  the  situation  does  not  seem  right,  relocate  the 
interview  or  ask  the  extra  people  to  leave.  Be  sure  that  such  a  request  cannot  be 
perceived  as  being  initiated  by  the  interviewee.  Do  not  allow  additional  people  to  join 
the  interview  once  it  has  started.  The  interview  team  must  be  cognizant  of  the  potential 
that  enemy  or  government  informants  may  attempt  to  hear  what  is  being  said.  An 
effective  way  to  start  the  interview  is  for  the  interviewer  to  introduce  him  or  herself  and 
to  explain  that  part  of  his  or  her  job  is  to  help  the  Afghan  Government  better  serve  the 
people.  Tell  the  interviewee  that  to  best  help  the  government  improve,  the  team  (or 
agency,  the  Coalition,  etc.)  must  know  what  the  people  really  need  and  how  the 
government  is  doing  for  them.  Ensure  the  interviewee  that  all  answers  will  be 
anonymous.  Each  interview  should  be  conducted  with  a  single  interviewee.  Even  if 
more  than  one  person  is  present  and  is  participating  in  the  conversation,  answers  must  be 
assigned  to  a  single  interviewee  and  recorded  as  such.  Do  not  conduct  two  interviews 
simultaneously  within  the  same  small  group.  As  a  minimum,  this  will  be  distracting  to 
the  interviewee.  A  likely  problem  with  conducting  simultaneous  interviews  is  that 
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interviewees  will  influenee  eaeh  other’s  answers.  During  the  eourse  of  the  interview,  the 
best  teehnique  is  for  the  interviewer  to  eonduet  the  interview  in  the  order  that  the  topies 
appear  on  the  AGQI  data  eolleetion  sheet.  Rehearsals  should  be  eondueted  by  eaeh  team 
to  ensure  that  eaeh  of  the  three  members  of  the  team  have  exaetly  the  same  understanding 
of  the  meaning  of  eaeh  statement  on  the  AGQI  data  eolleetion  sheet. 

There  were  initial  eoneerns  about  how  interviewees  might  respond  to  seeing  someone 
reeording  their  responses  and  asking  questions  aeeording  to  a  form.  Based  on  experienee 
to  date,  it  does  not  appear  that  the  obvious  use  of  the  AGQI  data  eolleetion  sheet  in  any 
way  detraets  from  the  quality  of  the  interview.  In  faet,  it  appears  that  the  obvious  use  of 
the  form  reinforees  to  the  interviewee  that  the  interviewer  eonsiders  his  or  her  opinions 
important.  This  may  differ  with  different  populations. 

When  eompleting  the  data  eolleetion  sheet,  it  is  important  to  eomplete  eaeh  question. 
However,  if  the  interviewee  does  not  seem  to  understand  the  question  after  a  few  attempts 
at  rephrasing,  it  is  best  to  skip  the  question.  Do  not  eonvey  frustration  or  disappointment 
to  the  interviewee.  Remain  interested  and  appreeiative  of  his  or  her  honest  opinions  and 
for  taking  the  time  to  speak  with  you.  Eaeh  question  must  have  only  one  answer  on  the  0 
to  5  seale.  The  interview  team  must  make  the  judgment.  If  in  doubt,  make  a  note  and 
then  finalize  the  single  answer  as  soon  as  possible  after  the  interview. 

5.5.3.  Experienee  with  the  AGQI  to  Date.  The  following  observations  are  based  on 
experienees  with  the  final  version  of  the  AGQI  data  eolleetion  sheet  as  employed  by  the 
Parwan,  Jalalabad,  Gardez,  and  Kandahar  PRTs.  Some  interpreters  have  initially  been 
skeptieal  about  the  level  of  honesty  that  interviewees  would  display.  After  partieipating 
in  AGQI  interviews,  many  of  these  same  interpreters  have  been  impressed  with  the  level 
of  honesty  they  pereeived  among  the  interviewees.  In  general,  the  interpreters  that  we 
have  worked  with  in  performing  these  interviews  have  expressed  that  the  proeess  is  very 
valuable  and  important.  The  responses  from  interviewees  have  been  positive,  with 
interviewees  frequently  stating  that  they  were  glad  that  someone  was  finally  asking  them 
what  they  thought.  After  eondueting  35  AGQI  at  the  Jalalabad  PRT  (data  to  be  presented 
elsewhere),  a  group  of  10  eivil  affairs  soldiers  and  interpreters  were  asked  their  opinions 
about  the  AGQI.  The  eivil  affairs  soldiers  had  over  five  months  experienee  on  the 
ground  in  Afghanistan  at  the  time.  Responses  are  shown  in  table  4.  Responses  were  on  a 
1  to  5  seale  with  a  seore  of  1  meaning  strongly  disagree  and  a  seore  5  meaning  strongly 
agree;  in  the  table,  1  and  2  were  grouped  as  disagree  and  4  and  5  were  grouped  as  agree. 
The  responses  indieate  that  the  AGQI  proeess  was  very  useful  for  PRT  operations,  helped 
the  teams  learn  more  about  the  population,  and  was  well  worth  the  time  required.  There 
was  no  systematie  evidenee  that  the  interviewees  were  either  more  or  less  honest  due  to 
the  presenee  of  Coalition  troops.  Aneedotally,  interpreters  at  eaeh  loeation  eommented 
that  people  were  apt  to  be  more  honest  and  open  with  Coalition  military  personnel  than 
with  loeal  government  offieials.  At  all  PRTs  where  the  AGQI  proeess  has  been 
employed,  ineluding  Gardez,  Jalalabad,  Kandahar,  and  Parwan,  the  general  feeling  was 
that  the  AGQI  interview  proeess,  in  addition  to  providing  data  for  use  in  monitoring 
MOE,  provides  a  diseussion  format  that  leads  to  an  enhaneed  understanding  of  the 
population  on  the  part  of  the  personnel  working  in  the  area  and  eondueting  AGQI.  It  was 
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noted  that  with  most  other  interactions  with  the  populace,  discussions  are  limited  to  a 
single  topic,  such  as  a  well  project,  and  do  not  include  the  in-depth  discussions  of  the 
population’s  needs  and  attitudes  that  are  facilitated  by  the  AGQI. 

5.6.  Utility  of  AGQI 

The  AGQI  may  be  employed  according  to  a  national  strategy  (see  Section  6)  or  used  in  a 
targeted  manner  by  PRT,  other  civil  affairs  or  psychological  operations  elements,  or  by  a 
variety  of  multilateral  partners.  The  data  from  AGQI  are  useful  in  guiding  decisions  on 
CMO  activities  at  unit  through  national  levels,  guiding  good  governance  mentorship 
programs,  and  monitoring  progress  over  time  in  a  statistically  valid  manner.  The  AGQI 
is  a  tool  that  empowers  civil  affairs  and  tactical  psychological  operations  teams,  as  well 
as  interagency  personnel,  in  better  understanding  the  population  and  its  needs. 
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Table  4.  Post-AGQI  Survey  of  10  Experienced  Civil  Affairs  Team-A  Soldiers  and 
Interpreters _ _ _ _ 


Statement 

Disagree 

Neutral 

Agree 

Time  well  spent  (mean=36 
min) 

0 

0 

10 

AGQI  is  more  useful  than 
AIMS  village  assessment 

0 

1 

9 

AGQI  helps  you  learn  a  lot 
about  the  population  that 
you  did  not  know 

0 

2 

8 

People  are  more  truthful 
when  talking  with  Coalition 
military 

2 

6 

2 

People  are  very  willing  to 
talk  to  military 

1 

2 

7 

AGQI  is  the  single  most 
productive  PRT  activity  to 
learn  about  the  population 
and  its  needs 

0 

2 

8 
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6,  National  Sampling  Strategy 

6.1,  General 

A  systematic  means  for  monitoring  governance  quality  is  required  so  that  progress  in 
governance  development  can  be  determined  over  the  long  term.  We  have  developed  two 
complimentary  types  of  assessments  of  governance,  the  AGQI  and  the  AGM.  This 
describes  the  rationale  and  basic  approach  to  sampling  using  these  two  assessments. 

6.2,  Sampling  Approach 

Three  distinct  approaches  were  considered  in  determining  the  sampling  strategy  to  be 
used  to  monitor  development  of  good  governance  in  Afghanistan.  The  first  was  to 
monitor  governance  in  the  capital  city,  Kabul.  Benefits  of  this  approach  include:  1) 
simplicity,  2)  relatively  small  resource  requirements,  and  3)  the  ability  to  monitor 
development  of  the  central  government.  Limitations  of  this  approach  include:  1)  lack  of 
representation  of  other  geographic  regions,  2)  lack  of  representation  of  ethnic  groups  in 
other  parts  of  the  country,  and  3)  the  distinctly  different  nature  of  Kabul  compared  to  the 
rest  of  the  country,  due  to  the  extensive  ISAF,  Coalition,  and  other  international  presence. 

The  next  potential  approach  considered  was  to  monitor  governance  in  all  geographic 
regions  of  the  country,  for  example  in  each  district.  The  advantages  of  this  approach 
include:  1)  representation  of  all  geographic  areas,  2)  representation  of  all  ethnic  groups, 
and  3)  representation  of  urban  areas,  remote  rural  areas,  and  all  gradations  in  between. 
The  limitations  of  this  approach  are:  1)  it  is  the  most  resource  intensive  approach  in  terms 
of  funding,  personnel,  equipment,  and  time;  2)  inability,  considering  realistic  resource 
limitations,  to  obtain  enough  samples  from  any  given  area  to  be  able  to  draw  many 
important  conclusions;  and  3)  large  resource  requirements  would  reduce  time  and  other 
resources  available  to  engage  in  governance  mentoring  relationships. 

The  third  approach  considered  was  the  Indicator  Cities  approach.  In  this  approach, 
selected  cities  would  be  monitored.  The  assumption  would  be  made  that  changes  in 
governance  in  these  cities  would  be  representative  of  Afghan  governance  progress 
overall.  The  advantages  of  this  approach  include:  1)  ability  to  focus  on  specifically 
selected  cities  representing  all  regions  of  the  country,  2)  significantly  reduced  resource 
requirements  compared  to  the  second  approach,  3)  ability  to  obtain  enough  data  from 
each  city  to  allow  meaningful  interpretation  by  city  and  region,  and  4)  compatibility  of 
the  approach  with  governance  mentoring  relationships.  Limitations  include:  1)  lack  of 
representation  of  remote  rural  areas  and  2)  more  resource  intensive  than  the  first 
approach.  We  selected  the  Indicator  Cities  approach. 

6.3,  Selection  of  Indicator  Cities  and  Provinces 

Twenty-one  Indicator  Cities  were  selected  based  on  the  following  criteria  in  decreasing 
order  of  importance:  1)  population,  2)  transportation  routes,  3)  proximity  to  the  “Ring” 
Road,  and  4)  historical  importance.  The  provinces  in  which  these  cities  are  located  are 
considered  Indicator  Provinces  (Table  5;  Figure  3).  Each  PRT  will  be  assigned 
responsibility  for  Indicator  Cities  located  within  its  respective  PRT  region.  Each  PRT 
will  have  one  to  three  Indicator  Cities  (Table  6). 
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6.4,  Two  Types  of  Governance  Sampling 

Two  types  of  data  will  be  eolleeted  on  a  periodic  basis  as  a  means  to  monitor  progress  in 
the  development  of  good  governance:  AGQI  and  AGM.  The  AGQI  is  performed  by 
interview  and  involves  completion  of  a  data  collection  sheet  by  the  interviewer.  This  is 
an  assessment  of  governance  quality  from  the  point  of  view  of  the  consumers  of 
government.  Assessments  of  Governance  Quality  Indicators  will  be  performed  monthly 
for  each  Indicator  City  (Table  7).  These  assessments  will  also  be  employed  in  a  more 
intensive  manner  to  rapidly  develop  baseline  data  (Table  7),  for  example,  at  the  start  of  a 
formal  governance  mentoring  relationship.  In  addition,  AGQI  will  be  used  in  a  targeted 
manner  at  the  discretion  of  PRT  commanders  or  interagency  partners.  The  AGQI  data 
from  Indicator  Cities  will  be  considered  as  representative  of  both  the  cities  and  the 
respective  provinces.  Assessment  of  Governance  Quality  Indicators  data  will  be 
collected  regardless  of  whether  AGM  data  are  collected  for  any  given  city. 

Baseline  and  periodic  AGQI  will  be  performed  according  to  the  following  guidelines. 

1 .  Prior  to  sampling,  each  city  will  be  divided  into  three  geographic  sectors. 

2.  Each  month  for  the  assigned  Indicator  Cities,  3  average  citizens,  3  business  people, 
and  3  low-level  government  or  traditional  leaders  will  be  interviewed,  with  one  of  each 
category  representing  each  city  sector. 

3.  No  individual  may  be  interviewed  more  frequently  than  once  per  3  months. 

4.  The  two  major  ethnic  groups  (or  tribal  affiliations)  in  each  respective  city  will  be 
sampled  equally  during  any  3  month  period,  with  at  least  one  of  each  of  the  two  ethnic 
groups  represented  in  each  citizen  category  each  month. 

The  AGM  involves  an  in-depth  review  of  the  systems,  mechanisms,  and  functions  of 
government  offices  and  organizations.  It  must  be  conducted  by  personnel  with 
specialized  skills.  Assessments  of  Governance  Mechanisms  will  be  performed  for  both 
the  Indicator  City  government  and  the  Indicator  Provincial  government  once  every  three 
months,  for  Indicator  Cities  and  Indicator  Provinces  that  are  on  a  formal  good  governance 
development  program  (Table  7).  When  compiled  appropriately.  Indicator  City  and 
Indicator  Province  data  will  be  considered  as  representative  of  governance  development 
for  respective  regions  and  the  nation  as  a  whole. 

6.5,  Statistical  Basis  for  Sampling  Approach 

6.5.1.  Influence  of  Interviewer  on  AGQI  Data.  Some  organizations  have  expressed 
concern  that  the  mere  presence  of  military  personnel  during  the  AGQI  interview  process 
will  affect  responses,  rendering  the  data  unreliable.  While  it  is  true  that  interview  data 
may  be  affected  by  the  military  presence,  the  fact  is  that  there  is  an  effect  of  any  assessor 
(person  collecting  data).  There  is  an  effect  of  a  military  presence,  an  effect  of  the 
presence  of  civilian  coalition  government  representatives,  and  an  effect  of  the  presence  of 
NGO  personnel.  There  are  also  effects  of  subconscious  assessor  bias  and  many  other 
factors.  All  data  of  this  type  are  subject  to  assessor/interviewer  effects.  This  does  not 
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mean  that  the  data  are  not  valid  and  useful;  it  means  that  these  factors  must  be  understood 
up  front.  Each  measurement  has  an  unavoidable  “human  nature”  component  of  variance 
associated  with  the  interviewer.  Recognizing  this,  we  will  employ  the  following  methods 
to  account  for  and  minimize  these  effects.  These  methods  include:  1)  use  of  skilled 
assessors;  2)  use  of  a  standardized,  easy-to-use  tool;  3)  use  of  composite  indicators;  4) 
use  of  multiple  assessor-types;  5)  accounting  for  assessor  effects  statistically  (both  within 
and  between  assessor-types);  and  6)  use  of  a  systematic  sampling  plan.  There  is  no 
reason  to  believe  that  AGQI  data  collected  by  military  personnel  will  be  any  less  useful 
than  interview  data  collected  by  anyone  else. 

6.5.2.  General  Approach  to  Statistical  Analysis.  Most  AGQI  data  are  based  on  a 
modified  Likert  scale  using  possible  scores  of  0-5  representing  the  degree  of  agreement 
or  support  for  the  statement  or  item  listed  in  the  AGQI  data  collection  sheet.  A  score  of  1 
corresponds  to  “strong  disagreement/opposition”  and  a  score  of  5  corresponds  to  “strong 
agreement/support.”  Scores  of  0  correspond  to  “I  do  not  know”  or  “Nonapplicable.”  For 
most  questions,  a  score  of  0  will  be  treated  as  no  answer  (missing  value).  In  some 
instances,  the  answer  “I  do  not  know”  is  in  itself  a  bad  outcome  (e.g.  when  asking  if  the 
person  believes  he  would  be  treated  fairly  by  the  police  if  arrested).  In  cases  where  “I  do 
not  know”  represents  a  bad  outcome,  a  score  of  2  (disagreement/opposition)  will  be 
assigned.  As  a  result,  the  scale  will  be  1-5  for  the  purposes  of  most  data  analyses. 
Composites  of  multiple  related  variables  will  be  formed  by  a  process  of  averaging  and 
weighting  to  assess  selected  components  of  governance.  Although  the  1-5  scale 
described  is  not  a  true  continuous  scale,  the  numeric  value  of  each  response  is  positively 
related  to  the  level  of  agreement  with  the  statement.  For  statements  and  questions 
phrased  in  the  negative  and  statements  for  which  “agree”  represents  a  bad  outcome, 
scores  will  be  converted.  The  result  will  be  that  a  score  of  4  is  always  better  than  3,  3  is 
better  than  2,  and  so  on.  Further,  mean  scores  will  be  important  and  meaningful  during 
data  interpretation.  Composite  scores  will  not  be  limited  to  the  1-5  discrete  scale  but  will 
assume  a  continuous  nature  with  possible  values  ranging  from  1.0  and  5.0.  Therefore,  the 
initial  data  analysis  approach  will  treat  these  data  as  continuous. 

It  will  also  be  important  to  examine  these  data  on  the  basis  of  categorizations  such  as 
“mildly  to  strongly  disagree”  and  “mildly  to  strongly  agree.”  For  these  analyses,  data 
will  be  considered  categorical,  and  nonparametric  statistical  methods  will  be  employed. 

6.5.3.  Sample  Size  Estimation.  Required  sample  size  for  continuous  data  was  estimated 
based  on  pilot  AGQI  data.  The  value  for  the  “Support  for  Government  Institutions” 
composite  variable  for  the  pilot  conducted  at  the  Kandahar  PRT  was  3.27  ±  0.70  (mean  ± 
standard  deviation),  based  on  22  observations.  This  result  was  used  as  an  estimate  of 
expected  variance  in  the  larger  sample  population.  Sample  size  calculation  was  made 
based  on  the  following  equation  for  estimating  sample  size  for  detecting  differences 
between  two  means. 

n  =  2{s)\Z^+Zjsf  /  {MDCf 
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We  wish  to  detect  differences  between  means  with  a=.05  and  a  power  of  0.80.  Required 
sample  sizes  for  three  differences  of  interest  are  shown  in  table  8. 

Sampling  for  AGQI  will  be  in  multiples  of  3  to  provide  for  a  balanced  design  with 
respect  to  the  citizen,  business  person,  and  leader  categories.  The  national  sampling 
strategy  will  allow  us  to  detect  a  change  of  slightly  less  than  1.0  unit  in  a  composite 
governance  variable  for  a  city  between  any  two  months,  assuming  variance  similar  to  that 
observed  in  the  pilot.  We  will  be  able  to  detect  a  change  smaller  than  0.75  units  between 
any  two  quarters  (n=27  for  a  city  in  a  3  month  period)  and  a  change  smaller  than  0.50 
units  between  any  two  six  month  periods  (n=54  for  a  city  in  a  6  month  period). 

Although  the  primary  determinants  of  sample  size  estimation  were  the  ability  to 
detect  changes  for  continuous  variables  and  a  balanced  design,  it  was  also  important  to 
have  adequate  power  for  categorical  variables.  The  proposed  sampling  plan  will  allow 
detection  of  a  26%  change  in  proportion  of  responses  in  the  category  “agree  or  strongly 
agree”  for  any  given  discrete  governance  variable,  with  a=.05  and  a  power  of  0.80,  when 
examining  data  for  a  given  city  for  any  two  six  month  periods.  A  26%  change  is  relevant 
and  important.  This  estimate  assumes  a  starting  proportion  of  40%  and  an  increase  to 
66%,  and  is  based  on  the  following  equation. 


n  =  {Zce+Zfif{piqi+p2q2)  /  ip2-pi) 


Data  analysis  at  the  national  level  will  have  a  great  deal  more  power.  This  sampling 
strategy  is  planned  to  make  data  useful  over  time  at  both  the  PRT  level  and  at  the  national 
(CJCMOTF)  level. 

The  AGM  will  be  used  only  within  the  context  of  governance  mentoring  relationships 
and  will  be  dependent  on  participation  in  a  formal  good  governance  development 
program.  Therefore,  sample  size  estimation  was  not  performed  for  the  AGM. 
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Figure  3.  Afghanistan  Key  Indieator  Cities 


KHOST- 

KHOWST 

i_i 

n 

GARDEZ- 

PAKTIA 

1 _ 1 

n 

GHAZNI- 

GHAZNI 

□ 

■ 

n 

SHARAN- 

PAKTIKA 

LASHKAR 

KANDAHAR- 

GAH- 

KANDAHAR 

HELMAND 

Central  Highland 
Capitol 
r~l  Northeastern 
r~l  Eastern 


30 


USACAPOC  Technical  Report 


Table  5.  List  of  21  Indicator  Cities  and  Provinces 


Rank 

City  Name 

Province  Name 

1 

Kabul 

Kabul 

2 

J-Bad 

Nangarhar 

3 

Gardez 

Paktia 

4 

Kandahar 

Kandahar 

5 

Herat 

Herat 

6 

MeS 

Balkh 

7 

Konduz 

Konduz 

8 

Bamian 

Bamian 

9 

Ghazni 

Ghazni 

10 

Thin  Kot 

Oruzgan 

11 

Qalat 

Zabul 

12 

Khost 

Khowst 

13 

Shiberghan 

Jowzjan 

14 

Farah 

Farah 

15 

Baghlan 

Baghlan 

16 

Sharan 

Paktika 

17 

Lashkar  Gah 

Helmand 

18 

Sari  Pul 

Sari  Pul 

19 

Qala-e-Naw 

Badghis 

20 

Mihtarlam 

Laghman 

21 

Asadabad 

Konar 
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Table  6.  PRT  Responsibilities  for  Indicator  Cities 


Indicator  City 

PRT 

Asadabad 

Asadabad 

Bamian 

Bamian 

Gardez 

Gardez 

Sharan 

Gardez/Ghazni 

Ghazni 

Ghazni 

Farah 

Herat 

Qala-e-Naw 

Herat 

Herat 

Herat 

Jalalabad 

Jalalabad 

Mihtarlam 

Jalalabad 

Kandahar 

Kandahar 

Tirin  Kot 

Kandahar 

Lashkar  Gah 

Kandahar 

Khost 

Khost 

Konduz 

Konduz 

Baghlan 

Konduz 

Masar-e-Sharif 

Masar-e-Sharif 

Shiberghan 

Masar-e-Sharif 

Sari  Pul 

Masar-e-Sharif 

Kabul 

Parwan 

Qalat 

Qalat 
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Table  7.  Assessment  Type  and  Frequency  for  Indicator  Cities  and  Provinces 


Assessment 

Type 

Location 

Subject  of 
Assessment 

Frequency 

Baseline 

Periodic 

AGQI* 

Indicator  City 

Average 

Citizen 

9  the  first 
month 

3  per  month 

AGQP 

Indicator  City 

Business 

Person 

9  the  first 
month 

3  per  month 

AGQI* 

Indicator  City 

Leader 

9  the  first 
month 

3  per  month 

AGM** 

Indicator  City 

Office  of 
Mayor 

1  the  first 
month 

1  per  quarter 

AGM** 

Indicator  Province 

Office  of 
Governor 

1  the  first 
month 

1  per  quarter 

*AGQI:  Assessment  of  Governance  Qua! 

dty  Indicators 

**AGM;  Assessment  of  Governance  Mechanisms;  only  performed  within  the  context  of  a 
formal  governance  promotion  program 
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Table  8.  Estimated  Sample  Size  for  Three  Differences 


Difference  to  Detect 

Estimated  Sample  Size 

0.50 

31 

0.75 

14 

1.00 

8 
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7.  Conclusion 

A  system  of  metrics  was  needed  to  assess  progress  in  achieving  desired  tactical  and 
operational  effects  toward  a  defined  CMO  end  state  in  Afghanistan.  Such  a  system 
would  be  of  value  within  the  CJCMOTF  and  also  have  potential  to  become  an  integral 
part  of  an  overarching  system  of  metrics  to  monitor  overall  progress  in  Afghanistan  and 
in  the  Global  War  of  Terrorism.  To  meet  this  need,  a  MOE  system  for  CJCMOTF  CMO 
in  Afghanistan  was  developed. 

There  are  two  general  types  of  metrics  that  may  be  employed,  MOE  and  MOP;  these 
are  often  confused.  Measures  of  effectiveness  may  be  contrasted  with  MOP.  While 
MOP  monitor  whether  or  not  tasks  are  performed,  MOE  provide  information  on  whether 
or  not  performance  of  tasks  yields  the  desired  effects.  Properly  designed  and 
implemented  MOE  provide  critical  information  on  whether  and  to  what  degree  operations 
have  the  intended  effects.  Assessment  of  effects  is  an  integral  part  of  the  commander’s 
(or  other  decision  maker’s)  decision  cycle,  and  has  application  in  both  military  and 
civilian-lead  operations. 

To  most  effectively  develop  MOE,  a  systematic  and  prospective  approach  must  be 
used.  It  is  possible,  and  perhaps  sometimes  necessary,  to  select  MOE  as  tasks  are  defined 
during  the  course  of  a  long-term  operation.  However,  this  reactive  method  of  MOE 
identification  has  important  drawbacks  and  should  be  avoided  when  possible.  A  specific 
sequence  of  steps  should  be  followed  in  developing  MOE,  and  the  MOE  thus  developed 
should  possess  specific  characteristics.  In  practice  it  may  be  necessary  to  use  an 
abbreviated  approach  to  MOE  development,  due  to  constraints  of  time  or  other  factors. 
However,  omission  of  any  step  in  the  defined  process  should  be  based  on  a  conscious 
decision.  The  two  major  components  to  the  process  are:  1)  End  state  definition  and 
identification  of  potential  effects  and  2)  MOE  selection  and  refinement.  It  is  extremely 
important  to  adequately  define  the  end  state  at  the  outset  and  to  base  MOE  development 
on  the  defined  end  state.  While  mission  requirements  or  the  general  approach  to 
operations  (e.g.  lines  of  operation)  may  change,  the  desired  end  state  is  not  likely  to 
change  much  if  at  all  during  a  campaign.  When  MOE  are  based  on  the  end  state,  as 
opposed  to  a  given  approach  (e.g.  lines  of  operation),  the  MOE  system  will  be  able  to 
adapt  to  changes  with  minimal  impact,  and  will  be  more  focused.  Although  the  specific 
application  of  the  MOE  development  process  presented  here  was  for  CMO,  the  MOE 
development  process  described  can  be  applied  to  any  type  of  operation. 

The  CMO  end  state  was  defined  with  the  active  participation  of  multiple  partners 
using  a  multilateral  approach,  and  with  thorough  consideration  given  the  previously 
defined  end  states  of  relevant  stakeholders.  This  multilateral  approach  to  end  state 
definition  and  to  MOE  development  as  a  whole  proved  extremely  valuable  and 
productive,  and  should  be  used  whenever  possible.  The  MOE  system  reported  here  has 
the  flexibility  to  adapt  to  changes  in  command  relationships  and  reporting  structures. 
Because  it  is  based  on  a  well-defined  end  state,  the  system  is  robust  enough  to  remain 
effective  with  minimal  adjustment  through  changes  in  situation,  mission,  lines  of 
operation,  participating  agencies,  and  other  factors. 
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A  significant  component  of  the  MOE  system  described  is  the  AGQI  data  collection 
sheet.  The  AGQI  is  an  assessment  of  the  quality  of  governance  from  the  point  of  view  of 
consumers  of  government.  It  takes  the  form  of  an  interview  and  is  primarily  condueted  at 
the  level  of  average  citizens,  business  people,  and  low  level  government  and  traditional 
leaders.  However,  the  AGQI  may  be  used  with  any  category  of  person.  The  AGQI  may 
be  employed  aecording  to  a  national  strategy  or  used  in  a  targeted  manner  by  PRT,  other 
civil  affairs  or  psychological  operations  elements,  or  by  a  variety  of  multilateral  partners. 
The  data  from  AGQI  are  useful  in  guiding  deeisions  on  CMO  activities  at  unit  through 
national  levels,  guiding  good  govemanee  mentorship  programs,  and  monitoring  progress 
over  time  in  a  statistically  valid  manner.  The  AGQI  is  a  tool  that  empowers  civil  affairs 
and  tactical  psychological  operations  teams,  as  well  as  interagency  personnel,  in  better 
understanding  the  population  and  its  needs.  Although  the  AGQI  was  speeifically 
designed  for  use  in  Afghanistan,  it  will  be  useful  for  CMO  and  civilian  relief  operations 
in  other  parts  of  the  world,  with  slight  modification. 

One  of  the  findings  of  the  pilot  studies  using  the  AGQI  was  that  the  Afghanistan 
Information  Management  System  (AIMS)  village  assessment  provides  little  important 
information  that  is  not  aequired  through  use  of  the  AGQI  data  colleetion  sheet. 

Therefore,  it  was  decided  that  the  AIMS  village  assessment  form  will  be  used  only  on  an 
as  needed  basis  by  CJCMOTF,  while  the  AGQI  data  collection  sheet  will  be  eompleted 
for  all  interviews  and  meetings  with  loeal  populaee  or  leaders.  This  finding  raises  the 
larger  issue  of  the  need  for  robust  assessment  tools  for  civil  affairs  and  psychological 
operations  personnel  to  assess  populations  and  their  needs  aeross  a  wide  range  of 
operations  and  environments.  The  AGQI  has  great  potential  for  use  as  a  standard 
assessment  method  to  augment  Taetics  Techniques,  and  Procedures  (TTP)  already  in  use 
by  Civil  Affairs  and  Psychological  Operations  Forces. 

Development  of  a  MOE  system  and  of  speeific  methods  or  tools  to  gather  MOE  data 
are  not  in  and  of  themselves  sufficient.  A  eomprehensive  sampling  strategy  must  guide 
data  colleetion.  The  strategy  selected  must  eonsider  factors  such  as  resources  available, 
cost,  representation  of  the  populaee,  and  other  factors,  as  well  as  statistieal  validity.  Onee 
a  strategy  is  seleeted  it  must  be  followed  over  time  to  obtain  optimal  results. 

The  approach  to  MOE  development  presented  here  ean  be  applied  generally  to  MOE 
development  for  a  variety  of  different  types  of  operations.  The  speeific  system  developed 
to  monitor  MOE  for  CMO  in  Afghanistan  has  the  potential  to  provide  statistically  valid 
and  useful  information  and  provides  a  system  of  metries  to  monitor  good  governance 
development  and  other  progress  toward  the  CMO  end  state  in  Afghanistan. 


36 


USACAPOC  Technical  Report 


References 

United  States  Joint  Forees  Command.  Effeets-based  assessment  and  planning;  the 
science  of  operational  art.  Working  paper  (13  May  2003). 

United  States  Department  of  the  Army.  Operations,  Field  Manual  3-0.  United  States 
Department  of  the  Army,  Washington,  DC.  (June  2001). 

United  States  Joint  Chiefs  of  Staff.  Joint  Doctrine  for  Civil-military  Operations,  Joint 
Publication  3-57.  United  States  Joint  Staff,  Washington,  DC.  (8  February  2001). 

Islamic  Transitional  State  of  Afghanistan.  Afghanistan;  rebuilding  our  nation.  (2002). 
Accessed  on  the  internet  at;  www.reliefnet.org. 

United  Nations  Development  Program.  Agreement  on  provisional  arrangements  in 
Afghanistan  pending  the  re-establishment  of  permanent  government  institutions.  (2002). 
Accessed  on  the  internet  at;  mirror.undp.org/afghanistan/bonnagreement.html. 

United  States  Department  of  State.  Political-military  plan  for  post-hostilities 
Afghanistan.  (19  April  2002). 

Sechrist  L.  and  Sidani  S.  Quantitative  and  qualitative  methods;  is  there  an  alternative? 
Evaluation  and  Program  Planning.  18;77-87  (1995). 

Bryman  A.  The  debate  about  quantitative  and  qualitative  research;  a  question  of  method 
or  epistemology.  British  Journal  of  Sociology.  35;75-92  (1984). 

Hentschel  J.  Distinguishing  between  types  of  data  and  methods  of  collecting  them. 
World  Bank  Policy  Working  Paper  (April  1998). 

World  Bank  International.  New  empirical  tools  for  anti-corruption  and  institutional 
reform;  survey  of  public  officials-Cambodia.  (1999a).  Accessed  on  the  internet  at; 

WWW,  worldbank.  ors/wbi/sovernance. 

World  Bank  International.  New  empirical  tools  for  anti-corruption  and  institutional 
reform;  household  survey,  Cambodia.  (1999b).  Accessed  on  the  internet  at; 

WWW,  worldbank.  ors/wbi/sovernance. 

World  Bank  International.  New  empirical  tools  for  anti-corruption  and  institutional 
reform;  Cambodia;  business  environment  and  governance.  (1999c).  Accessed  on  the 
internet  at;  www.  worldbank.  ors/wbi/sovernance. 

Center  for  Democracy  and  Governance.  Handbook  of  democracy  and  governance 
program  indicators.  United  States  Agency  for  International  Development,  Washington, 
DC.  (August  1998). 


37 


USACAPOC  Technical  Report 

Appendix  A,  Assessment  of  Governance  Quality  Indicators  Data  Collection  Sheet 


A-1 


USACAPOC  Technical  Report 


1.  Date: 

2,  Recorder: 

3.  CAT-A: 

4,  PRT:  □  Bamian(l)  □  Gardez(2)  □  Ghazni(3)  □  Herat(4)  □  Jalalabad(5) 

□  Kandahar(6)  □  Konduz(7)  □  MeS(8)  □  Parwan(9) 

□  Other(lO): 

5.  Village: 

6,  Village  Location  (grid  (preferred)  or  Lat- 
Long): 

7,  District: 

8,  Province: 

9.  Population  (list  family  information  or  total  population): 


a.  Number  Families: _ b.  Family  Size: _ c.  Total  Population: _ 

10,  Ethnic  Group  of  Person  Interviewed: 

□  Aimak(l)  □  Baloch(2)  □  Brahui(3)  □  Hazara(4)  □  Kirghiz(5)  □  Kuchie(6) 

□  Nuristani(7)  □  Pashtun(8)  □  Qizilibash(9)  □  Tajik(lO)  □  Turkman(l  1)  □  Uzbeck(12) 

□  Other(13): _ 

11,  Gender  of  person  interviewed:  □  Male(l)  □  Female(2) 

12,  Is  person  interviewed  a  business  owner/management  representative?: 

□  No(l)  □  Yes(2) _ 

13,  Is  person  interviewed  a  member  of  the  media?:  □  No(l)  □  Yes(2) 

14,  Is  person  interviewed  employed  by  the  government?:  □  No(l)  □  Yes(2) 

15,  Is  person  interviewed  a  leader/leader’s  representative?:  □  No(l)  □  Yes(2) 

(Includes  elected  and  appointed  delegate/representative  of  the  people) 


Government  Employee  Only  (i.e.  answered  #14  Yes) _ 

16,  Occupation  in  Service  of  Government:  □  Education(l) 

□  Elected/appointed  delegate/representative  of  the  people(2)  □  Health  Care(3) 

□  Individual  Contract  Service(4)  □  Judicial  (Court  System)(5)  □  Military(6)  □  Police(7) 

□  Revenue  (accepts  money  (e.g.  taxes,  fees,  etc.)  on  behalf  of  govemment)(8) 

□  Other  civil  service(9) 
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Leader/Leader  Representative  Only  (i.e.  answered  #15  Yes) _ 

17.  Level  of  Leadership  Position:  □  Local/Village/City(l)  □  District(2)  □  Province(3) 

□  National(4)  □  Military(5) 

18,  Type  of  Leader:  □Civil(l)  □  Education(2) 

□  Elected/appointed  delegate/representative  of  the  people(3)  □  Judicial  (Court  System)(4) 

□  Military  ANA(5)  □  Military  Not  ANA(6)  □  Ministerial  Representative(7)  □  Police(8) 

□  Religious(9)  □  Traditional(lO) 

□  Other(ll): _ 

19.  Position  of  Leader  (Mark  all  that  apply):  □  District  (Sub)govemor(l)  □  Judge(2) 

□  Mayor(3)  □  Military  Commander(4)  □  Mullah(5)  □Police  Chief/Commander(6) 

□  School  Principal(7)  □  Shura  Leader(8)  □  Village  Elder(9) 

□  Other(lO): _ 

20,  Name  of  Leader/Leader  Representative  (Optional): 


Attitudes 


Rate  the  person’s  attitude 
toward  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Negative 

Negative 

Neutral 

Positive 

Strongly 

Positive 

21,  ITGA  (Central  Government)  0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

22.  ANA 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

23.  Local  Government 
(for  leader  read  as  “next  higher 
government”) 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

24,  Local  Police 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

25.  Presence  of  Coalition  Troops  0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

26,  Relief  Community 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

27.  Other  ethnic  groups  in  area 

0  □ 

ID  2D 

-Eormat  Change - 

3  □ 

4  □ 

5  □ 

28,  Person  believes  that  Does  not  know  (0)  □  No  (1)  □  Yes  (5)  □ 
civilian  relief  community 
and  military  are  separate 
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Security 


Categorize  the  person’s  level 
of  agreement  (actual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

29,  ITGA  is  responsible  for 
security  efforts  in  area 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

30,  The  security  situation  is 
good;  there  are  no  security 
concerns 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

31,  The  security  situation  will 
worsen  if  Coalition  troops 
leave  the  area 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

32,  The  Afghan  people 
will  benefit  if  the  government 
disarms  the  militia  forces 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

33,  The  government’s 
disarmament  process 
will  succeed 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

34,  If  person  is  a  victim 
of  a  crime,  the  police 
will  help 

0  □ 

ID  2U 

-Format  Change - 

3  □ 

4  □ 

5  □ 

35,  There  is  a  police  station 
within  a  half  day’s  travel 
(by  normal  means) 

Does  not  know  (0)  □ 

No  (1)  □  Yes  (5)  □ 

Human  Rights 

Categorize  the  person’s  level 
of  agreement  (actual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

36,  Person  feels  he  could  file  OD  !□  20  30  40  50 

a  human  rights  complaint 

without  fear  of  reprisal 

- Format  Change - 

37,  Are  there  human  rights  concerns  Does  not  know  (0)  □  No  (1)  □  Yes  (5)  □ 
(government  mistreats  populace)  in  this  area? 
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Democracy 


Categorize  the  person’s  level 
of  agreement  (actual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

38,  CLJ  delegates  are 
representative  of  the  people 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

39,  CLJ  elections  will  be 
conducted  impartially  and 
without  reprisals 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

40,  The  CLJ  process  is 
legitimate 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

41,  National  election  authority 
will  be  impartial 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

42,  Person  believes  he  can  vote 
freely  and  without  reprisal 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

Government  Competency  in  Meeting  People’s  Needs 

43,  ITGA  is  responsible  for 
relief  efforts  in  area 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

44,  Government  is  addressing 
citizens’  needs 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

45,  Person  believes  that  he  has 
adequate  information  about 
key  aspects  of  government 
activity 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

46,  Corruption  is  a  serious 
problem  in  government 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

47,  People  are  free  to  say  and 
write  what  they  think  about  the 
government 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

48,  Person  believes  he  will  be 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

treated  fairly  if  arrested  or 
if  he  files  a  eomplaint  with  the 
police 
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Government  Competency  in  Meeting  People’s  Needs  (Continued) 


Categorize  the  person’s  level 
of  agreement  (actual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

49,  To  get  government  services 
it  helps  to  pay  the  government 
official  a  personal  fee  (bribe) 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

50,  Person  is  confident  that 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

his  economic  situation  will 
improve  (including  food  and 
shelter  if  applicable) 

- Format  Change- 


51.  Education  available  (0)  □  None  (1)0  Informal  only  (homes/mosques) 

and  accessible 

(2)  □  For  boys  only  (3)  □  For  boys  and  girls 


Top  Three  Needs 


52.  Top  3  needs  as  stated  by  the  person: 

53,  Top  3  needs  as  judged  by  the  recorder: 

1  □  Agricultural  supplies 

1  □  Agricultural  supplies 

2  □  Electricity 

2  □  Electricity 

3  □  Employment  opportunities 

3  □  Employment  opportunities 

4  □  Equipment/facilities  for  government 

4  □  Equipment/facilities  for  government 

5  □  Food 

5  □  Food 

6  □  Infrastructure  repairs  not  addressed  in 

6  □  Infrastructure  repairs  not  addressed  in 

other  categories 

other  categories 

7  □  Irrigation 

7  □  Irrigation 

8  □  Medical  facilities 

8  □  Medical  facilities 

9  □  Medical  personnel 

9  □  Medical  personnel 

ion  Pay  for  government  employees 

ion  Pay  for  government  employees 

11  □  Police 

1 1  □  Police 

12  □  Road/bridge 

12  □  Road/bridge 

13  □  Schools 

13  □  Schools 

14  □  Security  presence 

14  □  Security  presence 

15  □  Teachers 

15  □  Teachers 

16  0  Veterinary  care 

16  0  Veterinary  care 

17  □  Water  (clean  water  to  drink) 

17  0  Water  (clean  water  to  drink) 

18  □  Other: 

18  □  Other: 

If  person  is  a  leader,  go  to  question  54. 

If  person  is  a  business  person,  go  to  question  62. 
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Leader  Onh 


Categorize  the  person’s  level 
of  agreement  (aetual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

54.  Government  has  adequate 
offiees,  equipment  and  supplies 
(refers  to  the  person’s  level  of 
government) 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

55.  Taxes  are  eolleeted  at 
this  level  of  government 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

56.  Taxes  are  submitted  to 
higher  government 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

57.  To  get  government  serviees 
for  his  village  or  offiee  it  helps 
to  pay  the  higher  level 
official  a  personal  fee  (bribe) 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

58.  Some  higher  level  officials 
will  enact  regulations  that 
favor  those  willing  to  pay  (bribe 

0  □ 

) 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

59.  Government  employees  are 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

paid  regularly 

- Format  Change - 

60.  Direetion  of  government  (0)  □  None  (1)0  From  higher  only 
eommunieations  (last  30  days) 

(2)  □  To  higher  only  (3)  □  Two-way 

61.  There  is  a  eentral  Does  not  know  (0)  □  No  (1)  □  Yes  (5)  □ 
government  (ITGA)  representative 

present  at  this  level  of  government 
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Business  Owner  or  Management  Representative  Only _ 

62.  Number  of  employees: _ 

63.  Type  of  business: 

□  Agriculture(l)  □  Banking/Finance(2)  □  Commerce  (retail  or  wholesale)(3) 

□  Construction(4)  □  Food  processmg(5)  □  Manufacturmg(6)  □  Mining/Natural  Resources(7) 

□  Professional  (e.g.  physician,  lawyer,  etc.)(8)  □  Service(9) 

□  Transportation,  distribution,  storage  (e.g.  trucking,  etc.)(10) 

□  Other(ll): _ 


Categorize  the  person’s  level 
of  agreement  (actual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

64.  It  is  easy  to  get  information 
about  government  policies  and 
procedures 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

65.  Laws,  regulations,  and  fees  0  □ 

are  a  problem  for  person’s  business 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

66.  Government  protects 
property  rights 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

67.  It  is  safe  to  use  the  banking 
system  for  financial  assets 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

68.  To  get  government  licenses 
or  services  for  business  it 
helps  to  make  some 
“unofficial  payments”  (bribes) 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

69.  Some  higher  level  officials 
will  enact  regulations  that 
favor  those  who  make 
“unofficial  payments”  (bribes) 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

70.  Business  is  better  this  year 
than  last  year 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

71.  Person  expects  to  be  in 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

business  3  years  from  now 
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Business  Owner  or  Management  Representative  Only  (Continued) 


Categorize  the  person’s  level 
of  agreement  (aetual  or 
estimated)  with  the  following: 

N/Aor 

Knows 

Nothing 

About 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

72,  Person  has  hired  or  will  hire 
employees  this  year 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

73,  The  road  infrastrueture  is  a  0  □ 

problem  for  the  person’s  business 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

74,  Unoffieial  road  use  tolls  are  0  □ 

a  problem  for  the  person’s  business 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

75,  Loeal  warlords  are  a  0  □ 

problem  for  the  person’s  business 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

76,  Crime  is  a  problem 
for  the  person’s  business 

0  □ 

1  □ 

2  □ 

3  □ 

4  □ 

5  □ 

Comments 
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Appendix  B,  Assessment  of  Governance  Quality  Indicators  Assessor  Training 


B-1 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Assessment  of  Governance 
Quality  Indicators  (AQ^I): 


Assessor  Training 


March  2 


wJ 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Purpose 


This  training  program  is  designeclJo'TOin 
personnel  who  will  perform  AGQI 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Training  Components 


Overview  briefing  on  MOE  deveigjrflent 

Orientation  to  the  Assessment  of ' 
Governance  Quality  Indicators  d^aia 
collection  sheet 


Small  group  practical  exercis€^ 

Total  training  time:  apppQd/iately  2  hours 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Expectations  After  Training 


•  Task:  Perform  assessments  of 
governance  quality  indicators  > 

•  Conditions:  During  CMC  througl^ut 
Afghanistan  as  directed  by  comrhand 

it 

•  Standard:  Assessments  areqrfoperly 
completed  and  data  forwarded  to 
CJCMOTF  within  esta^Nslied  timelines 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


5 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 

CJCMOTF 


Afghanistan 


Purpos©  ! 

■  L 

•  Provide  an  overview  of  the  CJCMpTF 
program  to  develop  metrics  of  (good) 
governance  development  J 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Outline  < 

Perspective  on  PRT  Operations 
Measures  of  Effectiveness:  The 

Program  Description 

-  CMO  Endstate  Definition 
^Overall  MOE  Program 

-  Assessment  of  Governance 
(Tactical  Level  MOE) 

Surffmary  and  Condusion 


ality  Indicators 
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PRT  Concept 

PRT  concept  has  evolved 

-  Most  PRTs  newly  established  but  conceptHnatofmg 

PRT  is  an  Interagency  Entity  » 

-  Interagency  Personnel  (Greater  Civili^  Presence) 

-  Interagency  Mission,  Operations, ^Ponding 

-  Expert  mentoring  capability 

-  Includes  ITGA 


-Collaboration  with  UN  ^ 

No  longer  a  purely  nri|mtary  operation 
Militaly-ocuaapoFteflt^nables  other  components 
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Current  and  Future  PRT  Roles  (1  of  2) 

Relationships/Influence 

-  Platform  for  promotion  of  local  andfpgional 
(good)  governance  development 

-  Specialized  functions  based  on  \o6p\  need 
^nd  PRT  personnel  (Ag,  Ed,  efc? 

-  Mitigate  Green-on-Green 

Security  Presence  andA(^isory  Role 
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Current  and  Future  PRT  Roles  (2  of  2) 

•  Government  of  Afghanistan  Liaison  _ f 

•  Project  Management 

\ 

•  Emergency  communications  hul:^ 

•  MEDEVAC  site/MEDEVAC  cooi^ination 


Liaison  with  Combat  Forces 


Limited  Humanitarian  Assistance 
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Assessment  of  Effectiveness 
Important  in  All  System^, 


ASSESS 


Detect 
Decide  Deliver 

Assess 


J 


'Industrial  and  medical  quality 
improvement  programs 
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The  question  is  not  IF  we  should  ^^s^ess 
our  effectiveness,  it  is  a  question  of  HOW 
we  should  assess  our  effectiveness 
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CJCMOTF _ _ J 

The  Need 

•  Systematic  means  to  assess  progress  in  • 
achieving  effects  toward  a  defined  endffafe 

•  Commonly  measure  tasks  performed 

-schools  repaired,  meetings  held,  we|ls  drilled 

•  Possible  to  perform  a  task  butfipfattain  the 

desired  effects  ^ 

•  Measure  of  PerformanceJ^ks  Performed 

•  Measure  of  EffectiverfeS's:  Effects  Achieved 
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Measures  of  Effectiveness  Versus 

/ 

Performance  (1  of  2)  j  . 

•  A  primary  desired  effect  of  our  ope)?atiunsns 
development  of  good  governance 

•  Task  toward  achieving  effect:  Ccpfcluct 
meetings  as  part  of  a  mentorincppi'OQrain 

•  Measure  of  Performance  (lyfQP)  provides 
information  whether  tasks  dre  completed 

•  MOP:  Number  of  nn^et^n^ conducted 

...but  did  governance  improve? 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Measures  of  Effectiveness  yersus 
Performance  (2  of  2) 


Measure  of  Effectiveness  (MOE)prdvid^ 
information  on  whether  our  operations  are 
resulting  in  the  desired  effects  ^  the 
path  to  our  endstate 


MOE:  Indicators  that  provid^4fiformation 
on  changes  in  status  of  ke/functions  of 
government 
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MOE  Development  Pro  ti 


•  Program  to  develop  MOE  was  in  mated  in  x 


mid-July  2003. 

•  Four  major  components  \ 

-  1)  Civil-Military  (C-M)  EndstateD^nition 

^  2)  Develop  Assessment  of  Governance 
Quality  Indicators  (AGQI;  Taerti^al  Level  MOE) 

-  3)  Develop  Assessment  of  Governance 


Mechanisms  (Opei^ipnal  Level  MOE) 

-4)  Identify  Nationaf  Key  Program  Indicators 


19 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


Afg(ianistan 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


JTr^ 


UNCLASSIFIED 

CJCMOTF 


Afghanistan 


Milestones,  Missions,  and 
Endstates  of  Various 
Organizations  Fall  Along  a 
Continuum  Leading 
to  the  GoA  Endstate 


ANA  can  defeat  armed  threats 
to  the  Afghan  people  and 
government,  ensuring  the 
integrity  and  security  of  Afghanistan 
without  coalition  assistance 


Taliban 

Rule 


National  Elections 

Successful 

2003 

2006 

2004 

2009 

Constitution 

Ratified 

9 

• 

GoA  Vision: 
A  Prosperous, 
Secure, 
Afghanistan... 


Police  force  meets 
international  stand/lPds^^ 
for  community  grolicing 


Major  Coalition 
Combat  Forces 
Depart 
Afghanistan 
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C-M  Endstate 


•  The  set  of  Afghan  government  cagal^ities j' 
and  conditions  that  must  exist  suglTtfiat: 

-  major  coalition  combat  forces  can  ddpart 

-  the  Afghan  government  will  continue  to 
progress,  with  only  NGOs,  civilfaFHdoalition 
government  reps,  and  internatipnal  aid 

•  Is  a  required  interim  state  (on  the  pathway 

tol'  "  '  '  ^^tate  as  described 

by  f  Afghanistan 
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Refined  C-M  Endstate  Definition 

A  Legitimate  Afghan  Government  th 
demonstrates  1)  Competency;  2)mfluence; 

3)  Democracy  that  is  Sustainable, 
Participatory,  and  Rights-respe^ng,  and 

4) jCore  Deliverables  in  the  Areas  of 
Security,  Justice,  Education^^d 
Mechanisms  for  Economi^evelopment. 
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Governance 


Competency,  Influence,  DemcWacy-^ 
comprise  the  basics  of  good  , 


governance 

Governance:  Exercise  of  a^ority 
through  formal  and  informal  traditions 
and  institutions  for  the  cj&mmon  good. 
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(Good)  Governance  (1  /Of  2) 

•  Process  of  selecting,  monitoring  g(nd  r 
replacing  governments 

-Voice  and  external  accountability  ' 

-  Political  stability  and  lack  of  violence,  crime, 


and  terrorism 


•  Capacity  to  formulate  and  imj5tement 
sound  policies  and  deliver  public  services 

-  ss 

-  m 
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(Good)  Governance  {2,oi  2) 


Respect  of  citizens  and  the  state  fertRe 
institutions  that  govern  economic  and 
social  interactions  among  them 

-  Rule  of  law 

-  Control  of  corruption 
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Importance  of  Good  Governance 

•  Opportunities  for  corruption  enormous  in  r 
developing  countries 

•  Good  governance  promotes  economic 
development  (national  wealth)  bdi  not 
vice-versa 

•  Good  governance  will  be  th^nderpinning 
of  the  development  of  ai^timate  Afghan 
government 


27 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


28 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Governance  Quality  Indicators 


•  Tactical  Level  MOEs  (Assessments  of  Governance  Qu'aliJ;^ 

Indicators;  AGQI): 

-  Assessment  at  the  level  of  “consumers”  of  government 
(citizens,  business,  low  level  government)  ( 

-  Of  significant  use  in  assessment  phase  ofcommander’s 
decision  cycle  at  the  PRT  level 

-  Useful  for  multilateral  decision  makpjss^ 

-  Also  valuable  when  consolidated  sjl  higher  levels 

-  C^n^be  collected  rapidly 

-  Require  skills  commorylti^^  personnel,  civilian 
interagency  personnel]  and  relief  community  partners 
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Governance  Mechanisms 

Assessments  of  Governance  Mechanisms  /Operational 

Level  MOEs): 

-  Assessment  at  city/district  to 

-  Primarily  useful  in  the  assessment  phas6  of  the 
commander’s  decision  cycle  for  operations  at  the 
operational  (CJCMOTF)  level 

-  Cfseful  for  multilateral  decision  ma^rs 

-  Generally  more  complex  than  tactical  level  measures 

-  Require  in-depth  assessm^ntx^  personnel  with 
specialized  skills  in  vaj^ibaS  functions  of  government 

-  Valuable  tool  in  local  jand  regional  governance 


mi 


in 
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National  Key  Program  Indic^ators 

•  Data  on  status  of  national  level  prog|ranis  or  ^ 
activities  critical  to  C-M  endstate 

-  Security  status  (UNAMA,  CJ2)  > 

-  ANA  status  (OMC-A)  ^ 

-  Police  status  (Germany  and  US  EtoSf 

^  ♦ 

-  Judicial  system  status  (Italy) 

-  Democracy  (CLJ,  Elections;  UI^AMA) 

-  DDR=<UNAMA;  JapanV^^^^ 

-  Infrastructure  (USAIQ;  multiple  international  donors) 

-  OtfieiMaased^rKntefagency  partner  requirements 
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Assessment  of 

Assessment  of 

National 

Governance 

Governance 

Key  Program 

Quality  Indicators 

Mechanisms 

Indicators 

Afghanistan 


/.Information 
Flow 

lete  System) 


— Raw  Data  Flow 

— Analysis/Summary Flow 
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Tactical  Level  MOE  Development 

(1of3)  '  I  _ ^ 

•  Development  of  a  standardized 
collection  sheet  that  includes  Tactical  Level 
MOEs  addressing  subfunctions  i^ntified  in 
the  CMO  endstate  definition 

•  Useable  at  the  CAT-A  level  ^ 

•  Data  useful  at  both  the  Pj^and  CJCMOTF 
levets  (and  to  other  ©i^amzations) 
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Tactical  Level  MOE  Developmefit  (2  of  3) 


•  Small  working  group  developed  ipjti^T^ 

prototype  set  of  MOEs  > 

•  Prototype  reviewed  and  modifie(;f4y  a 
small  group  of  Bamian  PRT  irrefrtbers, 
including  the  Commander,  pefS  rep,  and 
experienced  CAT-A  leaden 

•  Version  1  data  collecti^n^eet was  used 
in  a  pilot  at  the  Bamian  PRT 
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Tactical  Level  MOE  Developmeht  (3  of  3) 

•  Based  on  results  of  the  Bamian  plloL==-=--^ 

version  2  of  the  data  collection  shiet  was 
developed  and  sent  to  all  PRTs  f^ 
feedback  \ 

•  Version  2  data  collection  sh^  was  used 
in  a  pilot  at  the  Kandahar 

•  Current  version  incorppfates  feedback  and 
has  undergone  adrfiohal  working  group 
and  interagency^view 
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Sampling  Considerations  (^1  of  2) 

Effect  of  ANY  assessor  (person  cLj 
data) 

-  Effect  of  military  presence  on  respc^ses 

-  Effect  of  presence  of  civilian  coalitfon 
--government  rep  on  responses 

-  Effect  of  NGO  presence  on  r^ponse 

-  Subconscious  assessor 
-etc. 
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Sampling  Considerations  (2  of  2) 

All  data  of  this  type  are  subject  to  assessor/inferviewer* 
effects 


Must  understand  up  front 

-  Each  measurement  has  a  “human  nature” ^mponent  of 
variance  associated  with  the  interviewer-^solutelv 
unavoidable 


Methods  to  minimize  and  account  for  el 

-  Use  of  skilled  assessors 

-  Standardized,  easy  to  use  toj 

-  Use  of  composite  indic^rg" 

-  Use  multiple  assessorjtypes  and  balance 

Ian 
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Importance  of  Skilled  Assessors 

Tactical  MOE  is  not  analogous  to  “person-(^ 
the-street”  opinion  polls 


Need  trained  and  locally  operating  personnel 

-  Understand  effects  of  their  presenc^^ 

-Skilled  at  building  relationships  'C^7itl<populace 

-  Understand  implications  of  answers  and  can  lead 
the  conversation 

-  Understand  local  cultur^^'nu^nces,  and  situation 

-  High  payoff  as  indivtdual  and  organizational 
ionshios  are ijferilt  with  populace  over  time 
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Role  of  Interpreters 


Interpreter  is  an  essential  part  of  tlpeteam 

Must  work  and  practice  as  part  of  the  team 

Must  understand  this  assessment  process 
tabest  relate  attitudes  and  ah^ers  to 
questions 
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Employment  of  Assessments  of 
Governance  Quality  Indicators  (T|AC  MO^) 

•  As  required  by  CJCMOTF  nation^t^^~^ 
sampling  strategy 

-  Specified  balance  of  average  citiz^s, 
business  people,  and  leaders 

•  As  a  tool  for  PRT  operationp=^ 

-  Assess  status  of  an  area 

-Pre  and  post  assessfo^^nTbr  selected  cities  in 
a  mentoring  relaticfnship 

leted  assessments  based  on  mission 
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standard  Village  Assessments 


•  Pilot  revealed  that  village  assessraeifis 

provide  little  useful  information  and  no 
important  information  that  is  not  ^quired 
through  use  of  the  MOE  data^coHection 
sheet  ^ 

•  AIMS  village  assessr  rm  will  be  used 


onty~on  an  as  neede( 
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Data  Reporting 

Immediate  (within  48  hours  of  en 
mission) 

-  Electronic:  embedded  in  SURER  oras  an 
email  attachment  (req  TBD) 

Follow-up:  Turn-in  original  §h^ets  for 
archiving  (as  possible  at  le^st  monthly) 

Transitioning  to  Acces^^^^  transmit  as 
Excel  file  (attemptjfo  put  in  GIS) 


43 


CJ5  (Plans)/POC  MAJ  Pusateri 


UNCLASSIFIED 


UNCLASSIFIED 


CJCMOTF 


Afghanistan 


Summary  and  Conclusion 

j  .  j 

PRT  is  an  interagency  entity  with  an  interagency  mission^ 

Major  focus  of  PRT  operations  is  local  and  re( 

(good)  governance 

I 

Assessment  of  Governance  Quality  Indicators  (TAG 
MOE)  will  enable  assessment  of  progress  iff4chieving 
effects  toward  both  the  (interim)  CMO  /snd^^ate  and  the 
ultimate  Afghan  endstate 

Data  collection  is  a  team  effort  involx^g  trained  CA 
soldiers,  Interagency  Partners,  and  mterpreters 

Data  will  be  useful  at  PRTj,aiC5i\feTF,  CJTF180,  CFG, 
and  higher  levels 
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Questions  or  Comm^r'its 
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CJCMOTF _ ^1 _ 

Data  Collection  Sheet  (1  of  2) 

•  standardized  form  with  sections  th^t  will  bej- 

-  completed  for  all  interviews 

-  completed  only  for  leaders  or  busines^  people 

•  Designed  to  maximize  “check  the^ock” 

•  Eight  Parts 

-  Part  1:  Demographics  (15-20  Questions) 

-  P^2:  Attitudes  (8  questjj 

-  Part  3:  Security  (7  quSsfions) 

-  Part  4:  Human  Riatus  (2  questions) 
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-  Part  5 

-  Part  6:  Government  Competency  in  (Meeting 
People’s  Needs  (1 1  questions)  ) 

-  Part  7:  Leaders  Only  (8  questipog)^ 

^art  8:  Business  People  Onl^j;f6) 

•  Average  citizen:  48  questions 

•  Leader:  61  questions,^/"^^ 

•  Business  person:  p3  questions 


Collection  Sheet  (2  of  2) 


:  Democracy  (5  questions) 
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Guidelines  (1  of  2), 

Fill  out  demographics  before  the 
interview/meeting  if  possible 

Work  and  practice  as  a  team: 

-  Leader 


Afghanistan 


-  Recorder 
-^Interpreter 

Provide  translated  forms  fo 


I 


re[er 


Complete  the  fornf  as  you  go-too  long  for 
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Guidelines  (2  of  2) 


Discuss  methods  used  by  others-leamand 
share  information  > 


Forward  data  to  CJCMOTF 

^electronically  in  the  SURER  wflhin^48  hours  of 
mission  completion  (attachmpnf  or 
embedded)  f 

-original  data  collection ^h%dts  for  archive 
within  30  days 
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Report  in  SURER, 


MOE  Data  Follows 

I.  20031104 

2.  Jones 

3.  13 

4.  6 

6.  XXX  XX  XXXX  XXXX 

II.  M 
16.  (I( 

43.  2 

etc. 
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Practical  Exercise. 

In  small  groups  work  as  teams 


Afghanistan 


Alternate  as  role  player  citizen,  busipes^ 
person,  or  leader  , 

When  possible,  use  interpreters  as^le  players 
and  conduct  interview  in  Dari  or-Pa^hto 

Use  other  languages  where  possible  to  perform 
interview  in  other  than  Englis 

Alterriate  positions  oiyteam 

Discuss  and  share  nnlethods/approaches 
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Training  Summary 

Understand: 


why  we  monitor  MOE 

how  the  Assessment  of  Governance^ Quality 
Indicators  relates  to  MOE 


-  importance  of  good  governanoe4rHachieving 
^the  CMO  endstate 

Be  able  to: 

-eonduct  an  assessrae^tlnt^rview 

-  properly  complete  joata  collection  sheet 
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